#ai 900 | Explore Tumblr posts and blogs

greenoperator · 2 years ago

Text

Microsoft Azure Fundamentals AI-900 (Part 6)

Microsoft Azure AI Fundamentals: Explore computer vision

An area of AI where software systems perceive the world visually, through cameras, images, and videos.

Computer vision is one of the core areas of AI

It focuses on what the computer can “see” and make sense of it

Azure resources for Computer vision

Computer Vision - use this if you’re not going to use any other cognitive services or if you want to track costs separately

Cognitive Services - general cognitive services resources include Computer vision along with other services.

Analyzing images with the computer vision service

Analyze an image evaluate objects that are detect

Generate human readable phrase or sentence that can describe what image is detected

If multiple phrases are created for an image, each will have an associated confidence score

Image descriptions are based on sets of thousands of recognizable objects used to suggest tags for an image

Tags are associated with the image as metadata and summarizes attributes of the image.

Similar to tagging, but it can identify common objects in the picture.

It draws a bounding box around the object with coordinates on the image.

It can identify commercial brands.

The service has an existing database of thousands of recognized logos

If a brand name is in the image, it returns a score of 0 to 1

Detects where faces are in an image

Draws a bounding box

Facial analysis capabilities exist because of the Face Service

It can detect age, mood, attributes, etc.

Currently limited set of categories.

Objects detected are compared to existing categories and it uses the best fit category

86 categories exist in the list

Celebrities

Landmarks

It can read printed and hand written content.

Detect image types - line drawing vs photo

Detect image color schemes - identify the dominant foreground color vs overall colors in an image

Genrate thumbnails

Moderate content - detect images with adult content, violent or gory scenes

Classify images with the Custom Vision Service

Image classification is a technique where the object in an image is being classified

You need data that consists of features and labels

Digital images are made up of an array of pixel values. These are used as features to train the model based on known image classes

Most modern image classification solutions are based on deep learning techniques.

They use Convolutional neural Networks (CNNS) to uncover patterns in the pixels to a particular class.

Model Training

To train a model you must upload images to a training resource and label them with class labels

Custom Vision Portal is the application where the training occurs in

Additionally it can use Custom Vision service programming languages-specific SDKs

Model Evaluation

Precision - percentage of the class predictions made by the model that are correct

Recall - percentage of the class predictions the model identified correctly

Average Precision - Overall metric using precision and recall

Detect objects in images with the Custom Vision service

The class of each object identified

The probability score of the object classification

The coordinates of a bounding box of each object.

Requires training the object detection model, you must tag the classes and bounding box coordinates in a training set of images

This can be time consuming, but the Custom Vision portal makes this straightforward

The portal will suggest areas of the image where discrete objects are detected and you can add a class label

It also has Smart Tagging, where it suggests classes and bounding boxes to use for training

Precision - percentage of the class predictions made by the model that are correct

Recall - percentage of the class predictions the model identified correctly

Mean Average Precision (mAP) - Overall metric using precision and recall across all classes

Detect and analyze faces with the Face Service

Involves identifying regions of an image that contain a human face

It returns a bounding box that form a rectangle around the face

Moving beyond face detection, some algorithms return other information like facial landmarks (nose, eyes, eyebrows, lips, etc)

Facial landmarks can be used as features to train a model.

Another application of facial analysis. Used to train ML models to identify known individuals from their facial features.

More generally known as facial recognition

Requires multiple images of the person you want to recognize

Security - to build security applications and is used more and more no mobile devices

Social Media - use to automatically tag people and friends in photos.

Intelligent Monitoring - to monitor a persons face, for example when they are driving to determine where they are looking

Advertising - analyze faces in an image to direct advertisements to an appropriate demographic audience

Missing persons - use public camera systems with facial recognition to identify if a person is a missing person

Identity validation - use at port of entry kiosks to allow access/special entry permit

Blur - how blurry the face is

Exposure - aspects such as underexposed or over exposed and applies to the face in the image not overall image exposure

Glasses - if the person has glasses on

Head pose - face orientation in 3d space

Noise - visual noise in the image.

Occlusion - determines if any objects cover the face

Read text with the Computer Vision service

Submit an image to the API and get an operation ID

Use the operation ID to check status

When it’s completed get the result.

Pages - one for each page of text and orientation and page size

Lines - the lines of text on a page

Words - the words in a line of text including a bounding box and the text itself

Analyze receipts with the Form recognizer service

Matching field names to values

Processing tables of data

Identifying specific types of field, such as date, telephone number, addresses, totals, and other

Images must be JPEG, PNG, BMP, PDF, TIFF

File size < 50 MB

Image size between 50x50 pixels and 10000x10000 pixels

PDF documents no larger than 17 inches x 17 inches

You can train it with your own data

It just requires 5 samples to train it

Microsoft Azure AI Fundamentals: Explore decision support

Monitoring blood pressure

Evaluating mean tie between failures for hardware products

Part of the decision services category

Can be used with REST API

Sensitivity parameter is from 1 to 99

Anomalies are values outside expected values or ranges of values

The sensitivity boundary can be configured when making the API call

It uses a boundary, set as a sensitivity value, to create the upper and lower boundaries for anomaly detection

Calculated using concepts known as expectedValue, upperMargin, lowerMargin

If a value exceeds either boundary, then it is an anomaly

upperBoundary = expectedValue + (100-marginScale) * upperMargin

The service accepts data in JSON format.

It supports a maximum of 8640 data points. Break this down into smaller requests to improve the performance.

When to use Anomaly Detector

Process the algorithm against an entire set of data at one time

It creates a model based on your complete data set and the finds anomalies

Uses streaming data by comparing previously seen dat points to the last datapoint to determine if your latest one is an anomaly.

Model is created using the data points you send and determines if the current point is an anomaly.

Microsoft Azure AI Fundamentals: Explore natural language processing

Analyze Text with the Language Service

Used to describe solutions that involve extracting information from large volumes of unstructured data.

Analyzing text is a process to evaluate different aspects of a document or phrase, to gain insights about that text.

Text Analytics Techniques

Interpret words like “power”, “powered”, and “powerful” as the same word.

Convert to tree like structures (Noun phrases)

Often used for sentiment analysis

Determine the language of a document or text

Perform sentiment analysis (positive or negative)

Extract key phrases from text to indicate key talking points

Identify and categorize entities (places, people, organizations, etc)

Get started with Text analysis

Language name

ISO 6391 language code

Score as a level of confidence n the language returned.

Evaluates text to return a sentiment score and labels for each sentence

Useful for detecting positive or negative sentiment

Classification is between 0 to 1 with 1 being most positive

A score of 0.5 is indeterminate sentiment.

The phrase doesn’t have sufficient information to determine the sentiment.

Mixing language content with the language you tell it will return 0.5 also

Key Phrase extraction

Used to determine the main talking points of a text or a document

Depending on the volume this can take longer, so you can use the key phrase extraction capabilities of the Language Service to summarize main points.

Key phrase extraction can provide context about the document or text

Entity Recognition

Person

Location

OrganizationQuantity

DateTime

URL

US-based phone number

IP address

Recognize and Synthesize Speech

Acoustic model - converts audio signal to phonemes (representation of specific sounds)

Language model - maps the phonemes to words using a statistical algorithm to predict the most probably sequence of words based on the phonemes

ability to generate spoken output

Usually converting text to speech

This process tokenizes the set to break it down into individual words, assign phonetic sounds to each word

It then breaks the phonetic transcription to prosodic units to create phonemes for the audio

Get started with speech on Azure

Use this for demos, presentations, or scenarios where a person is speaking

In real time it can translate to many lunges as it processes

Audio files with Shared access signature (SAS) URI can be used and results are received asynchronously.

Jobs will start executing within minutes, but no estimate is provided for when the job changes to running state

Used to convert text to speech

Voices can be selected that will vocalize the text

Custom voices can be developed

Voices are trained using neural networks to overcome limitations in speech synthesis with regards to intonation.

Translate Text and Speech

Where each word is translated to the corresponding word in the target language

This approach has issues. For example, a direct word to word translation may not exist or the literal translation may not be the correct meaning of the phrase

Machine learning has to also understand the semantic context of the translation.

This provides more accurate translation of the input phrase or phrases

Grammar, formal versus informal, colloquialism all need to be considered

Text and speech translation

Profanity filtering - remove or do not translate profamity

Selective translation - tag content that isn’t to be translated (brand names, code names, etc)

Speech to text - transcribe speech from an audio source to text format.

Text to speech - used to generate spoken audio from a text source

Speech translation - translate speech in one language to text or speech in another

Create a language model with Conversational language Understanding

A None intent exists.

This should be used when no intent has been identified and should provide a message to a user.

Getting started with Conversational Language Understanding

Authoring the model - Defining entities, intents, and utterances to use to train the model

Entity Prediction - using the model after it is published.

Define intents based on actions a user would want to perform

Each intent should include a variety of utterances as examples of how a user may express the intent

If the intent can be applied to multiple entities, include sample utterances for each potential entity.

Machine-Learned - learned by the model during training from context in the sample utterances you provide

List - Defined as a hierarchy of lists and sublists

RegEx - regular expression patterns

Pattern.any - entities used with patterns to define complex entities that may be hard to extract from sample utterances

After intents and entities are created you train the model.

Training is the process of using your sample utterances to teach the model to match natural language expressions that a user may say to probable intents and entities.

Training and testing are iterative processes

If the model does not match correctly, you create more utterances, retrain, and test.

When results are satisfactory, you can publish the model.

Client applications can use the model by using and endpoint for the prediction resource

Build a bot with the Language Service and Azure Bot Service

Knowledge base of question and answer pairs. Usually some built-in natural language processing model to enable questions and can understand the semantic meaning

Bot service - to provide an interface to the knowledge base through one or more channels

Microsoft Azure AI Fundamentals: Explore knowledge mining

Used to describe solutions that involve extracting information from large volumes of unstructured data.

It has a services in Cognitive services to create a user-managed index.

The index can b meant for internal use only or shared with the public.

It can use other Cognitive Services capabilities to extract the information

What is Azure Cognitive Search?

Provides a programmable search engine build on Apache Lucene

Highly available platform with 99.9% uptime SLA for cloud and on-premise assets

Data from any source - accepts data form any source provided in JSON format with auto crawling support for selected data sources in Azure

Full text search and analysis - Offers full text search capabilities supporting both simple query and full Lucene query syntax

AI Powered search - has Cognitive AI capabilities built in for image and text analysis from raw content

Multi-lingual - offers linguistic analysis for 56 langues

Geo-enabled - supports geo-search filtered based on proximity to a physical location

Configurable user experience - it includes capabilities to improve the user experience (autocomplete, autosuggest, pagination, hit highlighting, etc)

Identify elements of a search solution

Folders with files,

Text in a database

Etc

Use a skillset to Define an enrichment pipeline

Key Phrase Extraction - uses a pre-trained model to detect important phrases based on term placement, linguistic rules, proximity to terms

Text Translation - pre-trained model to translate the input text into various languages for normalization or localization use cases

Image Analysis Skills - uses an image detection algorithm to identify the content of an image an generate a text description

Optical Character Recognition Skills - extract printed or handwritten text from images, photos, videos

Understand indexes

Index schema - index includes a definition of the structure of the data in the documents to read.

Index attributes - Each field in a document the index stores its name, the data type, supported behaviors (searchable, sortable, etc)

Best indexes use only the features that are required/needed

Use an indexer to build an index

Push method - JSON data is pushed into a search index via a REST API or a .NET SDK. Most flexible and with least restrictions

Pull method - Search service indexer pulls from popular Azure data sources and if necessary exports the Tinto JSON if its not already in that format

Use the pull method to load data with an indexer

Azure Cognitive search’s indexer is a crawler that extracts searchable text and metadata form an external Azure data source an populates a search index using field-to-field mapping between the data and the index.

Data import monitoring and verification

Indexers only import new or updated documents. It is normal to see zero documents indexed

Health information is displayed in a dashboard.

You can monitor the progress of the indexing

Making changes to an index

You need to drop and recreate indexes if you need to make changes to the field definitions

An approach to update your index without impacting your users is to create a new index with a new name

After importing data, switch to the new index.

Persist enriched data in a knowledge store

A knowledge store is persistent storage of enriched content.

The knowledge store is to store the data generated from Ai enrichment in a container.

#Microsoft #AI-900 #AI Fundamentals #AI/ML