Cribl, recognised for its AI Platform for Telemetry, has unveiled StreamAI, an enterprise AI gateway featuring a model router integrated into its open data platform. The new tool aims to effectively manage vast volumes of data, offering organisations enhanced control over AI traffic, usage tokens, and associated costs and risks.
Following an assessment of AI model performance on telemetry data, Cribl developed a model router to optimise AI inference processes. StreamAI channelises AI prompts and workloads to the appropriate model, ensuring cost-effectiveness while enhancing visibility and governance over AI operations. Furthermore, Cribl provides free inference for clients utilising its automatic data routing to verified AI models, thereby avoiding unexpected charges.
Optimising predictions and costs
With rising AI integration in business operations, predicting and managing expenditures on AI inference poses increasing challenges. Costs of infrastructure can surge unexpectedly, particularly when agents execute complex tasks with minimal human oversight. StreamAI offers real-time financial management by ensuring reliable AI inferencing across various proprietary and open-source models, setting budget caps and spending limits.
StreamAI offers real-time financial management by ensuring reliable AI inferencing
Clint Sharp, co-founder and CEO of Cribl, commented on the nuances of AI usage: “AI is not one-size-fits-all. The right model depends on the job, the context, and the economics of the request. Customers shouldn’t have to send every prompt to the most expensive model or build their AI strategy around a single provider. StreamAI gives them an intelligent control plane that routes each request to the model best suited to the work, helping reduce token costs while preserving the choice and flexibility to use the models they want.”
Model evaluation and financial efficiency
The model router utilised within StreamAI is based on Cribl’s SecIT Bench research, which analysed 20 AI models across 30 real-world IT and security scenarios. These scenarios included issues like security breaches and service outages.
The study highlighted a 17% gap in diagnostic accuracy against a 20-fold difference in investigative expenditure, suggesting that higher cost models are not invariably optimal for all tasks. In instances where AI-driven applications hit financial barriers, StreamAI automatically selects an alternate model to maintain functionality. With integrated circuit breakers, IT teams can control token usage and avoid exceeding budgetary limits.
Enhancing Security and Compliance
StreamAI is crafted to guide organisations in embracing AI without being tied to a singular model
StreamAI is engineered for enterprise deployment, embedding essential security and compliance measures within the routing process. It extends Cribl-Privacy's capabilities, the telemetry-focused model behind Cribl Guard’s background detection, attuned for high-volume, semi-structured data frequently encountered in telemetry. Security controls apply to every AI interaction, safeguarding sensitive data, thwarting malicious inputs, and regulating model access to prevent hazardous outputs.
StreamAI facilitates bidirectional redaction of sensitive data, protecting proprietary information, credentials, and sensitive content in outgoing prompts as well as responses. It also systematically records each model invocation and routing option as standardised, audit-ready telemetry, assuring complete transparency and evidential governance for all AI interactions.
StreamAI is crafted to guide organisations in embracing AI without being tied to a singular model, provider, or proprietary platform. The platform is set to be available shortly, with opportunities for early access available through Cribl account representatives.
