Introduction:
In today’s competitive business landscape, providing exceptional customer service is no longer optional – it’s essential. Stellapps stands out as the first of its kind startup in India, dedicated to the digitization of the dairy supply chain. Founded in 2011, it is an IIT Madras incubated, Bangalore based, Internet of Things (IoT) startup with a primary focus on data acquisition and machine learning. Milk, being the largest crop on this planet, highlights a strong demand for technology interventions, particularly in emerging markets where the yield per animal is low, traceability is inadequate, and quality is not up to the mark. Digitization of the Agri-Dairy supply chain in emerging markets is the area where Stellapps aids in unlocking unprecedented value on a very large scale.
Industry: Dairy
Offering: Gen AI
Business Need:
StellApps has an application the caters to farmers from rural India. The application helps farmers to get the detail for the farming related queries. Currently the existing application is not able to handle the native language features and sharing response only in English. As a result of this end-user of the application are not getting the desired benefit from this application. The StellApps team wants to enhance the application by introducing a feature that allows users to record audio questions in their native language. These questions relate to topics such as taking care of cattle, managing/handling the crops etc. In response, the application should provide response in the farmer’s local language with correct and accurate details.
Implementation:
Rapyder has proposed a solution powered by GenAI capabilities. We proposed solution using AWS Bedrock to answer the query from the existing documents. We are taking RAG based approach using Amazon RDS PostgreSQL as Vector DB and Amazon S3 as raw storage. We are creating a data pipeline by using services such as AWS Lambda, Amazon ECS with Fargate, AWS ECR and Amazon API Gateway to keep the data updated.
To build the GenAI based solution for the use case, there are two major activities that we need to address as part of the solution:
- Build Knowledge base for chatbot using PDF document
- Build the chat API that understands the user query and provide the accurate response
Farmers or villagers, the primary users, can ask questions in their local language by recording their voices. The audio is then translated to English using the Whisper API hosted on ECS Fargate. The translated text is used to search PGVector, extracting relevant information from documents. This information, along with RAG based approach using Bedrock, generates responses in English text. Stellapps own application uses the transcript from our solution to create the output in native farmers source language.
Solution Architecture:
The core of our GenAI solution is a microservices-based architecture deployed on the AWS cloud. This architecture provides scalability, flexibility, and resilience, ensuring that the chatbot can meet the demands of Stellapps.
Component:
- Knowledge Base pipeline:
-
- As per AWS best practices, we have kept raw data in S3 bucket. An AWS Lambda event will be triggered to pre-process the uploaded data.
-
- We are performing the embedding using Amazon Bedrock Titan model on the raw data, and ingesting the embeddings into Amazon RDS PGVector DB.
- GenAI Activity:
- We have created Whisper ECS API that accepts audio, and performs the transcription, and parse transcription to LLM API.
- We are converting the English input text into embeddings using Amazon Bedrock Titan model.
- We have implemented the data retrieval on VectorDB using similarity search algorithm using Langchain based approach.
- Response is getting created by using Claude2 model with help of Langchain based RAG pipeline using refined prompts after getting input from the StellApps team.
- As part of best practices, if the context or answer is not available for the given query chatbot is politely asking the end-user to connect with customer care and relevant field expert doctors.
LLM based tuning & improvement:
- Used chunking for the knowledge-based data while embedding data into PGVector.
- Experimented different “K” values while setting up RAG pipeline to get the better results while performing similarity for the query.
- Performed parameter tuning for TopP, token length and temperature control for getting the good accuracy.
- Added prompt engineering best practices and ensured that no false information is being provided as response and input is added to get the information vetted from the doctor or field experts.
AWS Services:
Reaping Rewards:
- Added feature to handle Indian languages with higher accuracy, this will help to get a larger Indian user-base.
- Customer is currently serving more than 100K farmers who are set to get benefitted from this solution.
- Reduced effort and maintenance cost for Knowledge base pipeline.
- Multi-lingual chatbot helped the business acquire more users and create more penetration across multiple geographical location serving with more than 15 languages across India. Customer is already acquired 17% new customer and growing
- As for many issue users were earlier visiting veterinary doctor, facing waiting time resulting in unsatisfactory outcome and spending cost for trivial issues. With this solution Customer team received positive feedback and appreciation for the integration and cost saved for the end-user.
