Generative AI in SaaS: Balancing Innovation and Costs

The rapid adoption of artificial intelligence, particularly large language models (LLMs), is revolutionizing the landscape of SaaS products. These advancements promise to enhance user experience and boost product adoption, but they also introduce new complexities in the cost of serving customers. This article delves into how generative AI could impact the Cost of Goods Sold (COGS) for SaaS products, exploring both the opportunities and challenges they present.

The Pre-Gen AI SaaS Era

Traditionally, the cost structure for SaaS products has been relatively straightforward. It includes:

Application Development (Engineering Cost)

Developing and maintaining a SaaS application involves significant engineering resources. This includes designing, coding, and testing the software to ensure it meets user needs and functions reliably.

Hosting (Compute, Storage, and Network)

SaaS products rely on robust hosting solutions to ensure availability, scalability, and performance. This encompasses the cost of servers, storage, and networking infrastructure.

Third party services

SaaS products tend to make use of several third party APIs. Typically the cost of APIs is absorbed in the SaaS subscription. 

Software Maintenance and Support

Ongoing maintenance is crucial to keep the software up-to-date with the latest features, security patches, and performance improvements. Customer support also plays a vital role in resolving user issues and ensuring satisfaction.

With this simple cost structure, majority of the SaaS products could offer the full product experience for a fixed subscription charge relying on usage-based fees. Companies enjoyed relatively high profit margin and did not need to consider usage based pricing. According to data presented by Sapphire Ventures, the profit margin of well-run SaaS companies was around 75% in 2023.

The question now is, with generative AI APIs being integrated into hundreds of human-product touchpoints, can SaaS companies still offer fixed subscription pricing while maintaining similar profit margins?

Post Gen-AI SaaS

If you are building a SaaS product in 2024 without incorporating generative AI, you are falling behind in the race. Generative AI has become a table stake. I am seeing companies and products adopting generative AI mainly for the following use cases:

Generate Content

Content generation has been the most versatile use case for generative AI algorithms. Tasks such as writing a blog post or sending an email are now being done by AI.

End-to-end workflows

With the rise in R&D of LLM agents, products are enabling people and enterprises to build end-to-end business workflows using AI. Workflows such as event planning and travel booking are now being automated.

Simplifying User Interaction

This is the most interesting use case to me personally. By leveraging LLMs and conversational AI, products can simply how users interact with the product. With conversational AI, users can perform tasks and navigate product features using natural language without navigating different screens, all under the same chatbot interface. This reduces the learning curve and enhances overall user experience. At Konigle, we are building a chatbot that acts as an assistant for users to interact with various features such as building a landing page, analysing site's traffic and creating marketing campaigns. All the tasks are done through a very simple chat interface. 

While simplifying user interaction might seem like a mere productivity enhancement, it has profound implications for product adoption. Users can interact with the product without needing to master its interface or features, making it more accessible and appealing.

Cost Implications

However, this convenience comes at a cost. For every user action intended, an AI agent must interpret the request and execute appropriate actions. This adds a new line item in the COGS, as maintaining and scaling AI agents require additional computational resources and sophisticated infrastructure. By  offering simplified user interaction capability, products incur hefty cost on the compute. This brings us to some key questions that SaaS companies should look into:

Key Considerations

Who Will Bear the Additional Cost?

Should the extra cost associated with LLM features be passed on to customers, or should it be absorbed into the existing subscription fee? If charged separately, could it inhibit product usage and adoption? Striking the right balance is crucial to ensure the new features add value without becoming a financial burden.

Cost Effective Use of AI models

How can SaaS providers deploy LLMs in a cost-effective manner? In my opinion, companies should focus on building on top of foundational models and hosting them in-house rather than just plugging in external APIs. This way, you have control over the infrastructure, and the cost of serving customers will become exponentially insignificant as foundational models become more effective. While this approach presents challenges in deploying AI algorithms at scale, I believe it is worth the effort.

Another possibility I foresee is built-in support for LLM APIs inside browsers. Smaller models that can run on-device should be sufficient for most use cases. Having support for these APIs within browser JavaScript APIs can greatly simplify the technology stack of modern AI-powered web applications. Apple has already shown the capability of smaller models to run on devices like iPhone 15.

Emerging Interaction Patterns

As LLMs become integral to SaaS products, new interaction patterns are likely to emerge. How will users' expectations and behaviors evolve? Understanding these changes will be key to designing intuitive and effective AI-driven interfaces.

Conclusion

The integration of LLMs into SaaS products presents a double-edged sword. On one hand, it promises to simplify user interactions and drive product adoption, making sophisticated software more accessible than ever. On the other hand, it introduces new costs and complexities that must be carefully managed.

As SaaS providers navigate this evolving landscape, they must thoughtfully consider how to balance the benefits of AI with the associated costs. The answers to these questions will shape the future of SaaS, determining how effectively these products can leverage AI to deliver value to users and maintain a sustainable business model.