Le projet THINK

Intel Labs presents latest research in Natural Language Processing at EMNLP.

In this tutorial we will learn how to write code that automatically detects what type of AI accelerator is installed on the machine (Gaudi, GPU or CPU), and make  the needed changes to run the code smoothly.

Intel Labs has set several works in motion to further the development and application of advanced artificial intelligence technologies to scientific challenges, particularly in the field of materials science.  

Nesime Tatbul is a senior research scientist in the Parallel Computing Lab at Intel Labs and acts as Intel’s lead PI for DSAIL. 

Highlights: 

Intel co-sponsors a new phase of the Data Systems and Artificial Intelligence Lab (DSAIL) university research program at the Massachusetts Institute of Technology (MIT). 
Over the next four years, DSAIL will generalize the vision of instance optimization to a wide variety of data systems and applications.  

A new phase of our Data Systems and Artificial Intelligence Lab (DSAIL) university research  program at the Massachusetts Institute of Technology (MIT) officially kicked off on October 20-21, 2022, during an annual meeting in Cambridge, MA. Established in 2018, the program pioneered Machine learning (ML) for data systems research, exploring the use of modern  ML techniques in improving the design and performance of large-scale data systems and applications. This includes enhancing or replacing key components of traditional data systems (e.g., index structures, scheduling algorithms, query optimizers) with their learned counterparts to allow them to adjust automatically to changing data distributions and query workloads. These learned components have been applied in novel use cases through joint projects with Intel, including ML-enhanced DNA sequence search and query optimization. Furthermore, the team built SageDB, an “instance-optimized” accelerator for the open-source PostgreSQL database, showing how these learned components can be integrated together in an end-to-end system that outperforms expert-tuned databases on analytical database workloads.  

“Through close collaboration with Intel and our corporate sponsors, we have been able to show that ML can be used to develop novel data systems that successfully adapt to the data, workloads, and hardware environments in which they operate and successfully integrated those systems into a number of real-world applications.” Sam Madden, DSAIL Co-Director and MIT College of Computing Distinguished Professor. 

Research Agenda 

One of the major thrusts of DSAIL’s continued research agenda is to build instance-optimized systems. These systems self-adjust to handle a workload with near-optimal performance under a given set of operating conditions as if built from scratch for that specific use case. Instance optimization is motivated by growing trends in  the variety of data-intensive applications and the heterogeneity of hardware/software platforms where they are being deployed. While specialized solutions can lead to better performance, manually developing and tuning them for each individual use case is not economically feasible. The team’s work to date has shown promise in leveraging ML to overcome this challenge. 

In recent years, there have been more endeavors to apply machine learning to algorithmic and system problems, many of which are driven by DSAIL. These works include ML applications ranging from video processing to storage layouts to log-structured merge trees and many other data management tasks. However, so far, most research has been focused on improving individual components. In this second phase of DSAIL, a key goal will be to investigate how learned components can be combined to build an entire, holistically instance-optimized system that does not require administrator intervention. In collaboration with co-sponsors Amazon, Google, and Intel, DSAIL will also generalize the vision of instance optimization to a wide variety of data systems and applications through novel designs across edge-to-cloud deployment settings. Examples include hybrid transactional/analytical processing (HTAP) systems, key-value stores, data lakes, and visual data analytics systems. In conjunction with common sense reasoning based on domain knowledge (e.g., represented as knowledge graphs or probabilistic models), ML techniques will continue to play a central role in the lab’s upcoming research agenda. 

Instance-Optimized Clouds  

Achieving instance optimization at the cloud scale introduces a new set of challenges and opportunities for research. The increasing complexity of cloud service infrastructures and their cost-performance tradeoffs are getting harder for cloud developers and users to navigate. More fundamentally, the disaggregation of data services in the cloud challenges the performance of traditional data system architectures due to their monolithic designs. In a joint vision paper published at the Conference on Innovative Data Systems Research (CIDR) earlier this year, Intel and MIT proposed a new metadata-rich cloud storage format called Self-organizing Data Containers (SDCs) to enable flexible data layouts that can self-adapt to client workloads. SDCs have three key properties that will enable automated performance optimizations in disaggregated database architectures:  

They flexibly support a variety of complex physical data layouts beyond simple column orientation via replication and partitioning. 
They explicitly represent rich metadata that can be used for optimizations, such as histograms and data access patterns. 
They can self-organize themselves over time as they are exposed to client query workloads.  

Preliminary experiments with real-world visual dashboarding applications indicate that even simple layout optimizations enabled by workload awareness of SDCs can achieve 3-10x speedups over traditional range partitioning. This work represents a foundational first step toward achieving instance optimization in modern cloud databases. 

Instance-Optimized Video Processing  

Video processing is a prime example of a data-intensive application domain that can substantially benefit from instance optimization. High volumes of video data are generated daily by a wide variety of applications, from social media to traffic monitoring. Applying state-of-the-art ML algorithms to efficiently analyze these datasets in real-world settings presents an interesting set of challenges and opportunities. Prior research by MIT DSAIL (e.g., MIRIS) and Intel Labs (e.g., VDMS) has demonstrated that there is potential for significant performance gains by tailoring these algorithms to the specific data and workload contexts that they are used in. Going forward, the DSAIL team will explore extending these efforts on multiple fronts to enable automated video search and analytics optimizations. 

For instance, Video Extract-Transform-Load (V-ETL) is one of the research problems that the DSAIL team is currently investigating in the context of large-scale video data warehouses. In order to prepare them for analytical queries, live video streams with varying content dynamics must be processed through user-defined ingestion pipelines that consist of expensive computer vision tasks, such as object detection and tracking. For resource and cost efficiency, there is a need for adaptive parameter tuning in such pipelines (e.g., frame rates, image resolutions, etc.) with changing video dynamics. The team is working on a novel approach that will continuously maintain high video content quality with a low cloud cost budget, even under peak load conditions. 

Given the success of the first phase of DSAIL, we are excited to support continued research in this area. This work has the potential to directly inform future design decisions within cloud data centers and enable a wide range of new applications. We look forward to jointly exploring these opportunities with our DSAIL collaborators at MIT and co-sponsors Amazon and Google. 

The Habana® team is excited to be at re:Invent 2022, November 28 – December 1.  We’re proud that Amazon EC2 DL1 instances featuring Habana Labs Gaudi deep learning accelerators are providing an alternative to GPU-based EC2 instances, delivering a new level of performance and efficiency in developing, training and deploying deep learning models and workloads.

OpenVINO™ has enabled automatic selection of the most suitable target device for AI inferencing

Highlights from Innovation 2022, panel discussion: “Productivity and Performance at Scale.”

Vy Vo is a research scientist working on deep neural networks in the Brain-Inspired Computing Lab as a part of Emergent AI research at Intel Labs.

Highlights: 

The Memory in Artificial and Real Intelligence (MemARI) workshop is scheduled for December 2nd, 2022, during NeurIPS 2022 at the New Orleans Convention Center. 

MemARI presents research on understanding how memory mechanisms can improve task performance in many different application domains, such as lifelong/continual learning, reinforcement learning, computer vision, and natural language processing. 
Intel contributed two of the 34 accepted workshop papers.  

This year at the Conference on Neural Information Processing Systems (NeurIPS), Intel Labs researchers, along with academic co-organizers from the Max Planck Institute for Software Systems, Princeton University, and The University of Texas at Austin, are hosting a workshop on Memory in Artificial and Real Intelligence (MemARI). The event will run from 8:30 AM – 5 PM on December 2nd, 2022, and include five keynote speakers, four paper spotlights, and additional poster sessions followed by a discussion panel moderated by Professor Ken Norman from Princeton.  

“Memory is essential to how we as humans make sense of the world,” says workshop co-organizer and professor Mariya Toneva at the Max Planck Institute for Software Systems. “Neuroscientists and psychologists have spent over a century learning about the brain “solution” for different types of memory. So should AI systems be informed by biological memory systems, and if so, how?” 

It may be the case that biological memory can address some of AI’s current problems. One of the key challenges for AI is to understand, predict, and model data over time. Pretrained networks should be able to temporally generalize or adapt to shifts in data distributions that occur over time. The current state-of-the-art (SOTA) in AI still struggles to model and understand data over long temporal durations. For example, SOTA models are limited to processing several seconds of video, and powerful transformer models are still fundamentally limited by their attention spans. On the other hand, humans and other biological systems can flexibly store and update information in memory to comprehend and manipulate multimodal streams of input. Cognitive neuroscientists propose doing so via the interaction of multiple memory systems with different neural mechanisms. Understanding how these systems interact to support our processing and decision-making in everyday environments is a deep topic of research in psychology, cognitive science, and neuroscience, with current advances driven by computational models that account for complex, naturalistic behavior. Indeed, understanding how information is stored and represented is key to creating knowledgeable models that can scale well in an economically feasible fashion. 

Several different types of memory also exist in artificial neural networks. Beyond the information stored in persistent, modifiable states, such as in long short-term memory (LSTM), memory is often manifested either as the static weights of a pre-trained network or as fast weight systems. Other approaches include infrequently updated external knowledge databases and memory-augmented neural networks (MANNs). New research on MANNs has made progress on the limitations of current ML models. However, an overarching understanding of how these types of memory can jointly contribute to better performance is still lacking. Thus, there are many questions yet to be answered:  

When is it useful to store information in states vs. weights vs. external databases? 
How should we design memory systems around these different storage mechanisms? 
How do they interact with one another, and how should they?  
Do the shortcomings of current models require some novel memory systems that retain information over different timescales or with different capacities or precision?  

These questions touch on many different application domains in AI, such as lifelong/continual learning, reinforcement learning, computer vision, and natural language processing. Furthermore, an understanding of memory processes in biological systems can advance research both within and across domains. This workshop will bring together researchers from these diverse fields to discuss recent advances, frame current questions and outstanding problems, and learn from the cognitive science and neuroscience of memory to propel this research. This kind of interdisciplinary gathering should produce valuable insights and help to elevate future research in this area.  

Read on for a preview of panel discussion topics.  

Panel Questions 

Memory is said to be important for generalization.  
Is memory augmentation necessary for out-of-domain generalization?  
What about temporal generalization / continual learning? 
Very large-scale models seem to already know a lot about the world.  
Does the number of parameters offset the need for memory augmentation? 
Classical computing systems separate storage and computation to scale each component independently.  
To what extent do biological systems and DNNs separate storage and computation?  
Can adding different forms of memory/storage enhance computation? 
Biological neural networks operate under a fixed size constraint, leading to properties like a tradeoff between memory capacity and accuracy. AI systems may not share these constraints.  
What constraints should we keep in mind when designing memory systems for AI? 

Scaling AI/ML deployments can be resource-limited and administratively complex while requiring expensive resources for hardware acceleration. Popular cloud platforms offer scalability and attractive tool sets, but those same tools often lock users in, limiting architectural and deployment choices. With Red Hat® OpenShift® Data Science (RHODS), data scientists and developers can rapidly develop, train, test, and iterate ML and DL models in a fully supported environment—without waiting for infrastructure provisioning. Red Hat OpenShift Service on AWS (ROSA) which is a turnkey application platform that provides a managed application platform service running natively on Amazon Web Services (AWS).

The AI for Accelerated Materials Design (AI4Mat) workshop is scheduled for December 2nd, 2022, during the NeurIPS 2022 at the New Orleans Convention Center.