Chip Huyen has conducted an analysis of the most popular open source AI software, revealing a stack consisting of four layers: infrastructure, model development, application development, and applications.
The infrastructure layer includes tools for serving, compute management, vector search, and databases.
The model development layer involves frameworks for modeling and training, inference optimization, dataset engineering, and evaluation.
Application development, also known as AI engineering, involves prompt engineering, RAG, and AI interface.
The most popular types of open-source AI applications are coding, workflow automation, and information aggregation.
Additionally, there is a category outside of the four layers for model repos, created by companies and researchers to share model code, such as CompVis/stable-diffusion, openai/whisper, and facebookresearch/llama.
Applications
The most popular types of applications are coding, bots (such as role-playing, WhatsApp bots, and Slack bots), and information aggregation.
AI engineering, which was a major focus in 2023, includes tools for prompt engineering, AI interface, agents, and AI engineering (AIE) frameworks.
Prompt engineering goes beyond just fiddling with prompts and covers areas such as constrained sampling (structured outputs) and long-term memory management.
AI interface provides an interface for end-users to interact with AI applications and includes web and desktop apps, browser extensions, bots, and plugins.
AIE frameworks are platforms that help developers build AI applications and often include tools for monitoring and evaluation.
The agent category includes tools for building sophisticated prompt engineering with constrained generation and plugin integration.