This evaluation shows materially higher concurrency and improved latency scaling when moving from a 64-core to a 96-core Intel® Xeon® configuration for Intel® AI for Enterprise RAG inference. The 96-core SKU doubles SLA-compliant concurrency for Llama-AWQ and Mistral-AWQ (32 → 64 users) across all workloads and increases Qwen-AWQ SLA concurrency by 33–50% (workload dependent) versus the 64-core system.
-
-
Articles récents
- Next-Gen AI Inference: Intel® Xeon® Processors Power Vision, NLP, and Recommender Workloads
- Document Summarization: Transforming Enterprise Content with Intel® AI for Enterprise RAG
- AutoRound Meets SGLang: Enabling Quantized Model Inference with AutoRound
- In-production AI Optimization Guide for Xeon: Search and Recommendation Use Case
- Argonne’s Aurora Supercomputer Helps Power Breakthrough Simulations of Quantum Materials
-
Neural networks news
Intel NN News
- Next-Gen AI Inference: Intel® Xeon® Processors Power Vision, NLP, and Recommender Workloads
Intel® Xeon® processors can deliver a CPU-first platform built for modern AI workloads without […]
- Document Summarization: Transforming Enterprise Content with Intel® AI for Enterprise RAG
Transform enterprise documents into insights with Document Summarization, optimized for Intel® […]
- AutoRound Meets SGLang: Enabling Quantized Model Inference with AutoRound
We are thrilled to announce an official collaboration between SGLang and AutoRound, enabling […]
- Next-Gen AI Inference: Intel® Xeon® Processors Power Vision, NLP, and Recommender Workloads
-