Talking Chatbot Text-to-Video Transformation: Innovations in GEN AI

• Developed a Generative AI application to enhance chatbot interactions by converting responses into compelling videos and hosted in Azure VM.
• Leveraged various open-source models, including adaptations from Huggingface, amalgamated and fine-tuned for optimal performance.
• Pioneered a multi-step approach: initiated by converting text responses into high-quality audio utilizing BARK an open-source audio model.
• Seamlessly synchronized audio with facial expressions by extracting facial landmarks, applying dynamic lip syncing, eyeblinking, etc.
• Integrated GFP GAN model as a face enhancer, elevating the visual appeal and expressions of the chatbot speaker.
• Spearheading the development of a comprehensive Processing Explanation video, seamlessly combining background visuals, dynamic speaker
• Presence with Pytorch 3D, and moving subtitles.
• Harnessing ChatGPT APIs for text summarization, empowering concise and impactful content generation, complemented by Stable Diffusion 2.1
• For crafting compelling background imagery.

Other Generative AI Research and Developments

• Proficient in utilizing Llama2 and various other LLA models, incorporating ChatGPT APIs seamlessly.
• Integrated MiniGPT4 successfully on cloud platform, enabling users to engage in Q&A interactions with input images.
• Demonstrated expertise in implementing diffusers and transformers, resulting in the creation of StableDiffusionImg2ImgPipeline. This innovative
• Pipeline facilitates image updates based on prompts.

Created multiple Virtual Assistant (Native Voice App available on Android and IOS both)

• Dialogflow was used to transform output into the user's speech and to receive input from the user's voice as input.
• Utilized different Rasa NLU and Action servers for each VA. used Flask as an endpoint for all virtual assistant communication.
• To improve the outcome, Entity Extractors, Lookup tables and Synomyms were used. Docker was utilised as a request endpoint.
• Deployed on Google Cloud via Kubernetes clusters, coupled with automated testing through GitHub Pipelines.
• Integrated into SkullCandy premium earphones, accessible via the Skull-IQ app as the virtual assistant named iHeart.

Created an end to end pipeline in Dataiku for production Deployment with multiple user's collaboration

• Currently Dataiku doesn't provide any direct option that multiple engineers can work simultaneously on different task and test them properly.
• Created pipeline where an engineer's trained model would be compared with production one and admin would get report mail.
• If admin seems fine with results then admin can provide approval to merge the changes and get the model updated.
• It enhanced the productivity of development by more than 20% with same workforce.

IOT enabled smart Refrigerator connected app

• Model was trained using Azure Machine Learning using data produced by IOT devices and sensors.
• Model forecasts if the device status is normal, critical, or something else.
• Used Azure app service to create Flask apis for live IOT data with status prediction and historical sensor data of refrigerator.

Custom MLOps platform

• Used ClearML as base part of our platform and deploy it on our AWS cloud platform as a service.
• Deploy GitLab on the same sever and integrated it with ClearML.
• Created GitLab CI/CD pipeline for model training, building, deployment, and testing in a new container. Sends admin report via email.

Other stuffs

• Used various Hugging face model pipelines and hosted different models into a cloud VM .
• Explored a variety of MLOps or related tools, such as MLFlow, Kubeflow, CML, DVC, etc.
• Worked on various AutoML tools, including H20AutoML, Auto-Keras, Auto-Sklearn, and AWS Sagemaker Studio.
• Experience on Auto EDA and Data preprocessing tools as well like Pandas Profiling, Sweetviz, Autoviz, Dataprep, etc

Categorize products on basis of product image and title:

• Built a model that was combination of CNN and RNN model to categorize the product.
• Used transfer learning with VGG16 to extract the context from the image and flatten that.
• Extracted the context from text input data through RNN model having Bi-directional LSTM layers.
• Combined both the models using Concatenate layer and got the categorical output.
• This model would be used in Government ecommerce marketplace to prevent fraud by saving a product in wrong category

Product part detection (Object detection):

• Seller previously used to offer incorrect information about a product that didn't exist in the final item.
• YOLO V5 was used to build a custom Computer Vision model that can detect object's component from an image.
• Labeling the images using Labelimg Library and trained them using YOLO V5. Identified the products having wrong description.

Created Rest APIs in a microservices based project using Flask, and worked with Virtual Machines, Docker containers.

Chatbot:

• Used libraries such as Spacy, Textblob, and NLTK, implemented several data pre-processing operations such as Stemming, Lemmatization and

Removing Stopwords. In addition, conducted a Sentiment Analysis.

• To get the best fit output, played around with the input text data across Rasa's Intent, Entity, Actions, and Stories then integrated it with Telegram.

Text Classification

• Create a system that automatically extracts new emails through IMAP and classifies them into useful and non-useful categories.
• Forged a classifier using NLTK for pre-processing, tokenizer, and embedding. Employed RNN model with multiple layers, utilizing Adam Optimizer

Experienced in API projects using Django Rest Framework and Django templates. Skilled in data processing with Pandas and NumPy.

Data Management & Data Mining:

• Acted web scrapping work using Beautiful-Soup, Selenium & Python Requests from different e-platforms.
• Use the web scrapped data for Competitive Price Analysis.

Data Analysis and Visualization

• Visualize the data using Plotly, Matplotlib and other tools.