Gandalf Insights: Revolutionizing Data Analysis with Natural Language Processing for All Users

Gandalf AI is an emerging company at the forefront of simplifying data analysis through advanced natural language processing (NLP) technology. Gandalf AI’s mission is to transform the way businesses interact with their data, making it easier to uncover trends, patterns, and actionable insights. Empower your data-driven decisions with Gandalf Insights from Gandalf AI, a product designed to enhance productivity by providing powerful analytics capabilities simply through a user-friendly chat interface.

Gandalf Insights is a data analysis product that allows users to analyze data from multiple databases using natural language queries. Even individuals without technical skills or knowledge can analyze data and generate reports like pie charts, bar charts, scatter charts etc using embedded visualization tools, simply by using natural language. That’s the power of Gandalf Insights. This feature makes data analysis accessible to many audiences and ensures the task is easy, fast, and efficient.

For instance, Users can ask questions in natural language, such as “What is the average sales per month?” or “Compare revenue by product category last year vs current year.”

Problem Statement

As many companies nowadays rely on data-driven decisions, executives often face significant challenges in analyzing data and making reports. The primary issues include the complexity of data querying, which traditionally requires certain technical expertise across various databases such as MySQL, PostgreSQL, and MongoDB. This may create a barrier for users from non-technical backgrounds, lagging in the expertise required to write and execute complex queries.

Integration issues also pose a problem, because many business firms use multiple databases, each storing distinct types of data. Thus, this clicked on an idea for a system that can efficiently link to multiple databases and handle a variety of data kinds. The lack of proper integration directly impacts the quality of decision-making by top-level management or executives. Because, executives and decision-makers need visual tools that can transform this raw data from multiple sources into clear, understandable graphs and charts. Consequently, there was a necessity for intuitive visualizations to represent complex data trends and insights which helps in quicker comprehension of trends, patterns, and anomalies, facilitating better and faster decision-making.

Solution

To overcome these challenges, Gandalf AI collaborated with Genese Solution to develop a product that takes data analysis and visualization to a new level with the power of Generative AI on AWS.

Gandalf Insights by Gandalf AI allows users to upload database information, such as schemas. These schemas are then transformed into embeddings using the embedding model of the AWS Bedrock service. Following the conversion, both schema and its corresponding embedding are stored in the powerful vector database provided by Amazon Web Services i.e. AWS OpenSearch Service.

Users can query their database using simple and plain natural language. This eliminates the need for advanced knowledge of SQL and databases. The AI engine underlying Gandalf Insights understands the user’s intent and translates it to an appropriate database query, making database querying accessible to users with no technical background.

In Addition, Gandalf Insights consists of a well-designed user interface. The application includes an advanced database management system that allows users to connect to their various databases. Users can then select a database and query it using plain natural language. Along with this, The application allows users to upload their data files, such as CSV, and query them immediately using Amazon Athena.

The well-designed user interface also allows users to view the database they have connected. Upon the selection of each database, users can also visualize tables that Gandalf Insights is currently connected to. This allows users to write their queries with the confidence that they will always have their results.

Gandalf Insights, with a powerful visualization tool, not only translates the user’s question but also generates easy-to-understand graphical representations, such as line graphs, bar charts, and pie charts based on the generated query. This allows users to interpret their data in a better way.

The Gandalf Insights also maintains conversation history. This allows users to go back to their analytics and get deeper insights into the data.

Large Language models that are currently available are prone to errors and hallucinations. Gandalf Insights minimizes these hallucinations through the use of advanced and iterative Chain of Thought prompting. This approach enhances the accuracy and reliability of the generated responses, ensuring more precise and dependable data analysis.

To further improve the accuracy of database query generation, Pre-trained models of AWS Bedrock have been fine-tuned to adapt powerful, pre-existing models to specific tasks. The user’s conversation history is used to fine-tune the model. It allows the model to minimize the error as much as possible by not repeating the same mistakes the user experienced previously.

Other than using Generative AI for database querying, Encryption, authorization and data recovery mechanisms have all been implemented to assure strong data security. AWS Backup facilitated automated backup scheduling and retention policies, ensuring data durability and easy restoration in case of undesirable events. Utilizing federated users and AWS Cognito, access is restricted to authorized organizational emails, enhancing security.

Result

Users across multiple departments, regardless of technical expertise, can now analyze data and generate insights, fostering a data-driven culture. The natural language querying feature has considerably shortened the time required for data analysis, allowing users to redirect their focus away from the mechanics of querying data and onto generating actionable insights and making data-driven decisions. With intuitive visualizations, users can quickly understand data trends and make informed decisions, enhancing overall strategic planning.

With a thorough survey, Gandalf AI concluded that the time spent on data analysis has been reduced by 40%, and 85% of non-technical staff reported increased confidence in using data for decision-making. The number of actionable insights generated per month has increased by 60%. Gandalf Insights’ users have expressed high satisfaction with the product, highlighting its ease of use and the ability to generate meaningful insights quickly. This positive feedback underscores the success of the product in meeting and exceeding user expectations.

With the help of AWS and its Generative AI services, Gandalf Insights has unlocked tremendous potential for strategic uses of data. It has proven how generative AI can simplify human-data interaction and make evidence-based decisions accessible for all.