#IntelLabs

[govindhtech]

How Nimble Models Make AI Accessible to All

Efficiency with Nimble Models

A continuous progression over a period of many years makes it feasible for generative artificial intelligence to become a prominent topic of discussion. In spite of the fact that a great number of enormous models with billions or even trillions of parameters were constructed, the reverse is true for smaller models with fewer than twenty billion parameters that might be just as accurate. Small and Nimble model: The Fast Path to GenAI was the title of a webinar that he organized in order to dissect this trend. Gadi Singer, Vice President of Intel Labs and Director of Emergent AI Research Lab, and Moshe Berchansky, Senior AI Researcher at Emergent AI Research Labs, Intel, were also present on the webinar.

These are the highlights of the webcast that he presents in this blog, and he encourages you to view the whole webcast so that you may have a better understanding of the content of the debate. Be sure to watch the really fascinating multi-modal generative AI demo that was created using a Llama 2 7B model. The video included the inclusion of the book Forest Gump as extra local material by means of Retrieval-augmented generation (RAG). Additionally, the show demonstrated how the model generated text from photos in addition to text from the book.

The Predicament Facing developers

There are a lot of options available to developers when it comes to generative artificial intelligence. A limited number of huge models are useful for broad and multi-purpose applications, whereas a large number of tiny models are useful for enhancing efficiency, accuracy, security, and traceability. The following considerations are necessary for constructing and designing generative artificial intelligence models:

Giant vs tiny, Nimble models (smaller by ten to one hundred times)

Open source models vs proprietary models

Generation that is retrieval-centric as opposed to retrieval-augmented

Types of models: general-purpose vs specialized and customized

The inference between cloud-based and local (on-premises, edge, or client)

As opposed to being tiny and Nimble model, giants were brawny

At this point in time, “small and Nimble model” refers to approximately anything that has less than 20 billion characteristics. The size criterion is a changing objective that may double in 2024; nonetheless, it provides a snapshot comparison versus 175 billion parameters for the ChatGPT 3.5 or more than a trillion for other systems. Scaling smaller models across an organization is more cost-effective than scaling larger ones because smaller models are simpler to change continually and run more quickly than larger ones.

It is worth noting that Dolly, Stable Diffusion, StarCoder, DALL·E, and Phi are all very effective models that operate at this scale. A recent demonstration of the remarkable gains that so-called “small language models” have made on benchmarks in terms of common sense, language comprehension, and logical reasoning was made by Microsoft Research’s Phi 2, which has 2.7 billion parameters. Such findings provide support for the idea that smaller models should play substantial roles, especially in mixed implementations alongside bigger ones.

Alternatives to open source software

In their article, Gadi and Moshe highlight the significance of open source in the development of GenAI models that are both compact and Nimble model. It was in February of 2023 when Meta launched LLaMA, which had models with 7 and 13 billion parameters respectively. It had a great deal of power, and it was first released as open-source software. In a short period of time, a series of animal-named models emerged, beginning with Alpaca, which was constructed on LLaMA by Stanford University, followed by Vicuna, which was developed by UC Berkeley, and then Falcon, Orca, and LLaMA 2 were all developed.

In comparison to what a single firm might do on its own, the quick, ongoing, and open growth of GenAI is far more impressive. Smaller models have caught up to several discrete benchmarks, despite the fact that GPT continues to be more powerful at a broad range of jobs.

In contrast to retrieval-centricity, retrieval-augmented

Data that has been trained using the model is essential for retrieval-centric models. Every single one of the initial versions of GPT-3 was dependent on the data that was stored inside the parametric memory of the GenAI model. This method is unable to take into account critical newer information, which might put the results of corporate operations at risk since it relies on information that is out of date.

This deficiency was addressed by the development of retrieval-augmented generation, often known as RAG. As a means of providing the model with more context, a retrieval front end makes use of numerous vector stores since it makes it possible to retrieve indexed and recent data. Because of this, the data that is entered is more verifiable and up to date, which results in findings that are more credible and also makes solutions more value.

General-purpose as opposed to specific and individualized

According to the conversations that are taking place with business clients about GenAI, they have seen an increase in the demand for specialized models that are adapted for particular functionality as opposed to the general wishes for a general-purpose, all-in-one model. Regarding the supply chain, for instance, a big healthcare provider posed the question, “Why would they want the same model to deal with their patient files as they do with their supply chain?” This is a valid concern, and the fine-tuning techniques that are already available and may be applied to more compact open-source models are an effective option.

The cloud against the local

When discussing the development of AI models, it is impossible to have a thorough discussion without taking into account the concerns around data security and privacy. Every business is required to give careful attention to these factors before sending data to a location where it is exposed to third parties outside the control of the company’s ownership. Keeping data local is made simpler by smaller models, regardless of whether they are operated on personal computers, private clouds, or any other platform.

Leveraging the tiny and Nimble model inflection point as a foundation

Currently, researchers at Intel Labs are working on a variety of enhancements for general artificial intelligence (GenAI) in the near future. These enhancements include efficient LLMs and the technologies that are required to support them.

Read more on Govindhtech.com

[hackgit]

[Media] ​​kAFL

​​kAFL A fuzzer for full VM kernel/driver targets. https://github.com/IntelLabs/kAFL #cybersecurity #infosec #linux

[debmalyashee]

It's taken 9 years to cross the door... 😊✌️ Behind The Lens : BK Cbe #NTTF #Intel #IntelLab #ArtificialIntelligent #Robotics #MachineLearning #Analytics #DataScience #IoT #hpc (at Electronic City) https://www.instagram.com/p/B0Faw1YJfEu/?igshid=b1jb0zudjo2h

[dieswaytoofast]

*"Hardware" vs "Software" - a cognitive gap*

Intel just open-sourced their Haskell Research Compiler (HRC) which has some amazing worn in it in proving performance, composability, and such-like.

Good news, right?

Well, not entirely. From the announcement, it is 

_"open sourced as is. We at Intel Labs are no longer actively working on this compiler"_

Which brings me to my point - the mindset difference between "chip-heads" and "software-heads". 

Chip designs have a very brief shelf-life, and, in many many ways, rewards short-term thinking. One doesn't do "chip upgrades" (°), and as a result, you tend to spend all your time getting to tape-out, bang in a last-minute patch, and you're done!

This is almost the polar opposite of software development, where you spend forever doing incremental updates in the face of ever-changing requirements (°°), and that's even before you start rolling out updates to customers. And this is even before we get to SaaS!

So yeah, if you're company is doing both, or, as in Intel's case, you're doing software in a hardware dominated org, expect some serious cognitive dissonance 🤕(°°°)

(°) Yeah, your CPU upgrades in the 90s don't count 😇

(°°) Agile FTW! 😄

(°°°) It's not that either if you is wrong, or right. It's that the economic imperatives are *different*

[hackernewsrobot]

HPAT – A compiler-based framework for big data in Python

https://github.com/IntelLabs/hpat Comments

[govindhtech]

Intel Labs AI Reference Kits for Next-Gen Health Tech

AI Reference Kits Features

With a concentration on AI/ML research, Dev Aryan Khanna is an Intel Student Ambassador for oneAPI. His most recent project was the Healthcare AI Reference Kits Companion, an intelligent healthcare application that combines text-based symptom analysis and image classification with cutting-edge deep learning models. The application aids in the early diagnosis of illnesses using X-ray pictures and symptom descriptions. For model optimization, it makes use of oneAPI libraries and Intel AI Reference Kits.

Concerning the Healthcare AI Companion Initiative

The intrinsic constraint of current healthcare solutions designed exclusively for particular GPUs served as the driving force behind the creation of the Healthcare AI Reference Kits Companion. This was a serious issue since it made it more difficult to adjust to Intel CPUs, which affected the customer base worldwide. By creating an intelligent healthcare tool that is designed for a variety of architectures and focuses on achieving effective performance on Intel hardware, the project seeks to close this gap.

In order to construct the Healthcare AI Reference Kits Companion, oneAPI tools with cross-architecture were utilized. With the use of these tools, developers may create single-language and platform apps that are easily adapted to and optimized for a variety of architectures, including CPUs from Intel. These instruments guarantee flexibility, effectiveness, and ease of use, which is consistent with the project’s objective of offering a complete and adjustable healthcare solution.

Use of Intel Software and Hardware Optimizations

Intel Deep Neural Network (oneDNN) Library via oneAPI

How: Showcase how oneDNN may increase CNN layers’ performance for image-based illness identification.

What: Convolutional layers are optimized by the library, which is essential for improving the effectiveness of illness identification in X-ray pictures.

When: During the training and inference phases of the project life cycle, in particular.

The Data Analytics Library (oneDAL) of Intel oneAPI

How: OneDAL was used to expedite feature engineering operations inside the data preprocessing pipeline.

What: Although not the main goal, oneDAL helps to improve feature engineering and guarantee data quality, which are essential for precise disease detection models.

When: Just before the model is trained, during the data preparation stage.

PyTorch Intel Extension

How: The comparison graph shows how the Intel Extension for PyTorch was able to accelerate model training. Mixed-precision training is also made possible by the PyTorch optimizations.

What: The Intel Extension for PyTorch allows for mixed-precision training without compromising accuracy, optimizes model training, and speeds up deep learning workloads on Intel CPUs.

when: Mainly while the model is being trained.

The Intel Student Ambassador’s Success Story

Indian undergraduate Dev Aryan Kanna attends Guru Gobind Singh Indraprastha University. His experience as an Intel Student Ambassador for oneAPI has been enlightening. As part of the Intel Student Ambassador Program, he has organized workshops, hackathons, and worked on projects that have improved his own learning and given the student attendees invaluable practical experience. With the program, he has access to a wealth of resources, including the newest Intel hardware, which has enhanced his knowledge of Intel technologies and stoked his love for creative problem-solving.

His ambition to use technology to better healthcare has been inspired by the Healthcare AI Reference Kits Companion project, which placed first runner-up in a recent Intel Student Ambassador Hackathon. He hopes to continue making a significant contribution to the nexus of artificial intelligence and healthcare with continued access to Intel’s capabilities.

Read more on Govindhtech.com

[govindhtech]

Intel Adds PCs AI Software and Hardware Developer Program

Intel AI pcs PCs AI Software As part of the AI PC Acceleration Program, Intel Corporation today announced the launch of two new artificial intelligence (AI) initiatives: the AI PC Developer Program and the inclusion of independent hardware manufacturers. These are significant turning points in Intel’s journey to empower the ecosystem of hardware and software to optimize and maximize AI on over 100 million Intel-based AI PCs by 2025.

The AI PC Developer Program is intended primarily to provide a seamless development experience and facilitate the large-scale adoption of innovative AI technologies by independent software suppliers (ISVs) and software developers. It gives users access to development kits that feature the newest Intel hardware, which includes the Intel Core Ultra CPU, as well as tools, processes, and frameworks for AI implementation.

Developers now have easy access to AI PC and client-focused toolkits, documentation, and training via the new developer resource website. The purpose of these compiled materials is to assist developers in optimizing AI and machine learning (ML) application performance and accelerating new use cases by fully using Intel Core Ultra CPU technology.

Developers who want to more about Intel’s worldwide partner network and how it is maximizing AI performance in the PC market should sign up for Intel’s AI PC Acceleration Program.

Independent hardware vendors (IHVs) now have the chance to get their hardware ready, optimized, and enabled for Intel AI PCs thanks to their inclusion in the AI PC Acceleration Program. Partners who meet the requirements may visit Intel’s Open Labs, where they can get co-engineering and technical assistance early on in the process of developing hardware solutions and platforms. Furthermore, Intel makes reference hardware available via this initiative to eligible IHV partners so they may test and enhance their technology in order to ensure optimal performance at launch.

The AI PC Accelerator Program has now onboarded 150 hardware providers worldwide, according to Matt King, senior director of Intel’s Client Hardware Ecosystem. “They can’t wait to expand their cutting-edge software and hardware solutions and share this momentum with their large, open developer community.”

The AI Acceleration Program for IHVs is open to developers and IHVs. In order to develop and elevate the AI PC experience to new heights, Intel is collaborating with its hardware partners. Come along with Intel as we accelerate innovation.

Why It Matters: AI will radically alter a wide range of facets of human existence, including creation, learning, employment, and relationships. By using Intel’s cutting-edge platform’s central processing units, neural processing units, and graphics processing units together with optimized software and hardware, anybody may take advantage of artificial intelligence with an AI PC. Intel works with a wide range of partners in an open ecosystem to provide improved performance, productivity, innovation, and creativity for end users. Intel is enabling ISVs and IHVs while spearheading innovations in the AI PC era.

Intel provides developers with extra value via various initiatives, such as:

Enhanced Compatibility: Developers can make sure their applications and software operate seamlessly on the newest Intel processors by having access to the most recent Intel Core Ultra development kits, optimization tools, and software. This improves compatibility and the overall end-user experience.

Performance Optimization: Software may be made more efficient and perform better if it is optimized for certain hardware architectures early in the development cycle. Better performance will be possible if AI PCs are broadly accessible thanks to this.

Global Scale and Increased Market Opportunities: Working with Intel and its large, open network of AI-enabled partners offers chances to grow your business internationally, penetrate new markets, and succeed in a variety of sectors.

With Intel Core Ultra processors spanning 230 designs from 12 worldwide original equipment manufacturers, Intel is bringing over 300 AI-accelerated capabilities to market by 2024 and provides a broad range of toolkits for AI developers to use.

About the AI PC Acceleration Program: Launched in October 2023, the program’s goal is to link independent software and hardware providers with Intel resources, such as training, co-engineering, software optimization, hardware, design resources, technical know-how, co-marketing, and sales opportunities.

PC Acceleration Program for AI Through the AI PC Acceleration Program, Intel will make artificial intelligence (AI) toolchains, training, co-engineering, software optimization, hardware, design resources, technical expertise, co-marketing, and sales opportunities available to independent hardware vendors (IHVs) and independent software vendors (ISVs).

Use Intel Core Ultra Processors on your PC to experience the power of AI. You might be able to increase your creativity, productivity, and security with the AI PC. We’re transferring AI apps from the cloud to PCs in response to market trends, enhancing privacy and lowering reliance on pricey data centers. Intel simplifies AI software development so you can concentrate on what really matters.

FAQS What is the AI PC Developer Program? A project by Intel to support AI technology research and adoption for personal computers. It goes after independent hardware vendors (IHVs), independent software developers, and ISVs.

Read more on Govindhtech.com

[govindhtech]

HoneyBee: Intel Labs and Mila’s Novel Language Model

Intel Labs Collaboration with Mila

By working together, Intel Labs and the Bang Liu group at Mila have developed HoneyBee, a cutting-edge large language model (LLM) specialized to materials science that is currently available on Hugging Face. This builds on Intel and the Mila – Quebec AI Institute’s ongoing research efforts to develop novel AI tools for materials discovery to address challenges like climate change and sustainable semiconductor manufacturing. A spotlight at the AI for Accelerated Materials Discovery (AI4Mat) Workshop at the Conference on Neural Information Processing Systems (NeurIPS 2023) and a Findings poster presentation at Empirical Methods in Natural Language Processing (EMNLP 2023) were both recently accepted for HoneyBee.

According to they partnership between Intel Labs and Mila on the MatSci-NLP article and blog, materials science is an intricate multidisciplinary area that aims to comprehend matter’s interaction in order to efficiently develop, create, and evaluate novel materials systems. The opportunity to develop specialized scientific LLMs that can comprehend specialized material, such chemical and mathematical formulas, as well as domain-specific scientific language is made possible by the abundance of research literature and textual information found in various documents. In order to accomplish this, they created HoneyBee, the first billion parameter-scale LLM that is open-source and focused on materials science. It has attained cutting-edge results on they MatSci-NLP benchmark, which is also open source.

Generate Reliable Training Data with MatSci-Instruct

A specific obstacle in creating LLMs in materials science is the deficiency of excellent annotations in scientific textual data. The difficulty is exacerbated by the fact that a large portion of scientific knowledge is expressed in language peculiar to a particular scientific field and has exact meanings within those contexts. Because high-quality data is so important, scientific LLMs need to assemble training and evaluation data using a reliable process.

Although expert annotation is the preferred method, it is not practical to execute at a large scale. They present MatSci-Instruct, a reliable instructions data generating procedure that may be used to produce fine-tuning data for LLMs in scientific domains, particularly materials science, in order to solve the difficulty of producing high-quality textual data. MatSci-Instruct expands upon two key realizations:

By assessing generated fine-tuning data using several, independent LLMs, they can reduce bias and add more resilience, resulting in trustworthiness for both the generated data and the final LLM.

Large-scale learning models (LLMs) have demonstrated emergent ability in domains where they were not trained at first, and instruction-based fine-tuning can help them become even more proficient in particular domains.

Iterative Improvement of Materials Science Linguistic Models

Generation: The Instructor’s (ChatGPT) materials science text data generation serves as the foundation for the LLM fine-tuning data.

Verification: Using predefined criteria, an impartial Verifier LLM (Claude) verifies the data generated by the instructor by removing low-quality data.

Fine-tuning and evaluating the models: HoneyBee language models are trained on verifiable data and subsequently assessed by the Evaluator (GPT-4), another independent LLM.

With each new cycle, the three previously mentioned procedures are iteratively performed to gradually enhance the performance of HoneyBee language models. With every modification, the quality of the HoneyBee LLMs and the resulting materials science text data both get better. To properly train LLMs on challenging scientific domains, the MatSci-Instruct produced data, as illustrated , covers a wide range of pertinent materials science topics.

Language Models for HoneyBees

They research describes several studies to gain a better understanding of the performance of HoneyBee and the usefulness of MatSci-Instruct. They begin by examining the relationship between the evaluations made by human experts and the verification outcomes from the Verifier and Evaluator models. Figure 3 illustrates the good agreement between the two techniques with a pretty high correlation between the evaluation by human experts and the LLMs. This implies that reliable fine-tuning data can be produced using the LLMs employed in the MatSci-Instruct procedure.

With every iteration of fine-tuning, HoneyBee-7b and HoneyBee-13b, which stand for the number of parameters in the LLM, exhibit increasing improvement. This offers proof of the iterative approach’ effectiveness.

Certain instances, indicated in light yellow, show that HoneyBee-13b can outperform the original Instructor (ChatGPT). Further evidence of the usefulness of MatSci-Instruct comes from the observation of similar behavior in other studies of instruction-fine-tuned LLMs.

Lastly, they examine how well HoneyBee language models perform using the MatSci-NLP benchmark (refer to Figure 5). Using the same methodology as outlined in the MatSci-NLP publication, they discover that HoneyBee performs better than all LLMs in the initial MatSci-NLP analysis. HoneyBee performs better than all LLMs in the zero-shot condition, when LLMs assess the benchmark data without further training, with the exception of GPT-4, which served as the Evaluator in MatSci-Instruct. However, HoneyBee-13b uses up to 10 times less parameters than other models and still achieves competitive performance with GPT-4. This illustrates how highly specialized HoneyBee is, which makes it a cutting-edge language model for materials science.

Read more on Govindhtech.com

[hackernewsrobot]

Intel Haskell Research Compiler

https://github.com/IntelLabs/flrc Comments