Share
Explore

Building the AI Model : Test Questions Bank


w25 : January 20 Session 3

Summarized Review Questions and Detailed Answers

Question 1:

What are the key differences between object-oriented programming and probabilistic programming, and how do these differences impact the design and development of AI models?
Summary of the Question:
This question asks students to compare two programming paradigms—object-oriented programming (OOP) and probabilistic programming (PP)—and explain how their differences influence the way AI models are designed and developed. It also touches on the statistical and mathematical foundations of AI, the challenges of working with different types of data, and the practical considerations for deploying AI models.
Detailed Answer:
Object-Oriented Programming (OOP):
Definition: OOP is a programming paradigm based on the concept of "objects," which are instances of classes. Objects encapsulate data (attributes) and behavior (methods).
Key Features:
Encapsulation: Data and methods are bundled together.
Inheritance: Classes can inherit properties and methods from other classes.
Polymorphism: Objects can take on multiple forms.
Impact on AI Development:
OOP is useful for structuring large-scale AI projects, such as defining reusable components (e.g., data loaders, model architectures, and evaluation pipelines).
It provides a modular approach, making it easier to maintain and extend AI systems.
Probabilistic Programming (PP):
Definition: PP is a paradigm that focuses on defining probabilistic models and reasoning about uncertainty. It uses statistical methods to infer relationships between variables.
Key Features:
Bayesian Inference: Models are built using probability distributions, and inference is performed to update beliefs based on data.
Flexibility: PP allows for the representation of complex, uncertain systems.
Impact on AI Development:
PP is essential for tasks involving uncertainty, such as natural language processing, recommendation systems, and decision-making under uncertainty.
It enables the use of probabilistic models like Bayesian networks and Markov chains.
Key Differences and Their Impact:
Focus:
OOP focuses on structuring code and reusability.
PP focuses on modeling uncertainty and reasoning about data.
Mathematical Foundations:
OOP relies on deterministic logic.
PP relies on statistical methods, such as Bayesian statistics and gradient descent.
Data Handling:
OOP is better suited for structured data.
PP excels in handling unstructured data, such as text and images, by leveraging tokenization and embeddings.
AI Model Design:
OOP is used to organize the components of AI systems.
PP is used to define the probabilistic relationships and optimize model parameters.
Practical Implications:
OOP is commonly used in frameworks like TensorFlow and PyTorch to define neural networks and training pipelines.
PP is used in probabilistic programming libraries like Pyro and Stan for tasks requiring uncertainty modeling.
Tools like Hugging Face and cloud platforms integrate both paradigms to deploy AI models effectively.

Question 2:

How do the concepts of tokens, weightings, and embeddings relate to the architecture and performance of AI language models? Explain the importance of these concepts and how they impact the training and deployment of AI applications.
Summary of the Question:
This question focuses on the fundamental building blocks of AI language models—tokens, weightings, and embeddings—and their role in model architecture and performance. It also asks students to explain how these concepts influence the training and deployment of AI applications.
Detailed Answer:
Tokens:
Definition: Tokens are the smallest units of text (e.g., words, subwords, or characters) that a language model processes.
Role in AI Models:
Tokenization splits text into manageable pieces for processing.
The choice of tokenization method (e.g., word-level, subword-level, or character-level) affects the model's ability to handle rare or unseen words.
Impact on Performance:
Proper tokenization ensures efficient representation of text and reduces the size of the vocabulary, improving training speed and accuracy.
Weightings:
Definition: Weightings are the numerical values assigned to the connections between neurons in a neural network. They determine the importance of each input feature or token.
Role in AI Models:
During training, the model adjusts weightings to minimize the loss function and improve predictions.
Weightings capture the relationships and affinities between tokens.
Impact on Performance:
Properly optimized weightings lead to better generalization and accuracy in language tasks.
Embeddings:
Definition: Embeddings are dense vector representations of tokens in a continuous space. They capture semantic and syntactic relationships between tokens.
Role in AI Models:
Embeddings map tokens to numerical vectors that can be processed by neural networks.
Pre-trained embeddings (e.g., Word2Vec, GloVe) provide a starting point for training, reducing the need for large datasets.
Impact on Performance:
High-quality embeddings improve the model's ability to understand context and relationships in text.
Architectural Implications:
Tokens, weightings, and embeddings are fundamental to the architecture of AI language models like transformers.
Neural networks use embeddings as input, process them through layers of neurons, and adjust weightings using gradient descent.
The architecture determines how well the model captures context and generates predictions.
Training and Deployment:
Training:
Tokenization and embeddings are critical for preparing data for training.
Weightings are optimized during training using algorithms like stochastic gradient descent.
Deployment:
Efficient tokenization and embeddings reduce the computational cost of inference.
Pre-trained models (e.g., BERT, GPT) leverage embeddings and weightings to provide state-of-the-art performance in real-world applications.

Question 3 (New Question with "First Day on the Job" Flavor):

Imagine you are tasked with building a chatbot for a customer service application. How would you use the concepts of tokenization, embeddings, and neural networks to design and train the chatbot? What challenges might you face, and how would you address them?
Summary of the Question:
This question asks students to apply their understanding of tokenization, embeddings, and neural networks to a practical task: building a chatbot. It also encourages them to think about potential challenges and solutions.
Detailed Answer:
Designing the Chatbot:
Tokenization:
Split user input into tokens (e.g., words or subwords) for processing.
Use subword tokenization (e.g., Byte Pair Encoding) to handle rare or unknown words.
Embeddings:
Convert tokens into dense vector representations using pre-trained embeddings (e.g., GloVe, FastText) or train custom embeddings.
Use embeddings to capture the semantic meaning of user input.
Neural Networks:
Use a transformer-based architecture (e.g., GPT or BERT) to process embeddings and generate responses.
Fine-tune the model on a dataset of customer service interactions.
Training the Chatbot:
Prepare a dataset of customer queries and responses.
Train the model using supervised learning, optimizing weightings with gradient descent.
Evaluate the model's performance using metrics like accuracy and BLEU score.
Challenges and Solutions:
Challenge: Handling ambiguous or incomplete user input.
Solution: Use context-aware models like transformers to understand the conversation history.
Challenge: Deploying the chatbot efficiently.
Solution: Use tools like Hugging Face Transformers and cloud platforms for scalable deployment.
Challenge: Ensuring the chatbot provides accurate and helpful responses.
Solution: Continuously fine-tune the model using feedback from real interactions.
This question encourages students to think critically about the practical application of AI concepts in a real-world scenario.



August 2
Table 1
Question
Correct Answer
Answer 1
Answer 2
Answer 3
Answer 4
Column 7
Explanation
Explanation 2
1
In the conceptual model for AI language models, which layer receives raw text?
1
Input Layer
Embedding Layer
Encoder Layer
Output Layer
Output Layer
The Input Layer is responsible for receiving raw text in the conceptual model.
The Input Layer is responsible for receiving raw text in the conceptual model.
2
What does the Embedding Layer do in an AI language model?
2
Generate output text
Convert words to numerical vectors
Extract patterns
Produce final text prediction
Produce final text prediction
The Embedding Layer converts words to numerical vectors, allowing the model to process text mathematically.
The Embedding Layer converts words to numerical vectors, allowing the model to process text mathematically.
3
Which layers are responsible for extracting patterns and relationships in the text?
3
Input Layers
Embedding Layers
Encoder Layers
Output Layers
Output Layers
Encoder Layers are responsible for extracting patterns and relationships from the input text.
Encoder Layers are responsible for extracting patterns and relationships from the input text.
4
In the provided code example, what type of neural network layer is used for the encoder and decoder?
3
Dense
Embedding
LSTM
Convolutional
Convolutional
The code example uses LSTM (Long Short-Term Memory) layers for both the encoder and decoder.
The code example uses LSTM (Long Short-Term Memory) layers for both the encoder and decoder.
5
What is the purpose of the output layer in the LanguageModel class?
4
Receive input
Generate embeddings
Process sequences
Produce final text prediction
Produce final text prediction
The output layer (Dense layer) in the LanguageModel class produces the final text prediction.
The output layer (Dense layer) in the LanguageModel class produces the final text prediction.
6
How is modular design in AI language models similar to Java programming?
1
Each layer is a self-contained unit
Layers are written in Java
Models use Java Virtual Machine
AI models are objects
AI models are objects
Modular design in AI models is similar to Java as each layer is a self-contained unit with a specific purpose, like Java objects.
Modular design in AI models is similar to Java as each layer is a self-contained unit with a specific purpose, like Java objects.
7
How does data flow in an AI language model compare to Java object interactions?
3
Through SQL queries
Via HTTP requests
Through the layers, similar to method calls
Using global variables
Using global variables
Data flows through the layers in AI models, similar to method calls between Java objects.
Data flows through the layers in AI models, similar to method calls between Java objects.
8
What concept in AI models is analogous to shared fields in Java objects?
1
The model's weights
The input data
The output predictions
The layer names
The layer names
The model's weights are like shared fields in Java objects, updated during training.
The model's weights are like shared fields in Java objects, updated during training.
9
How is the concept of composition applied in AI language models?
1
Layers are composed together to form the complete model
Models are composed of Java classes
Composition is not used in AI models
Models are composed of SQL tables
Models are composed of SQL tables
Layers are composed together to form the complete model, analogous to object composition in Java.
Layers are composed together to form the complete model, analogous to object composition in Java.
10
What OOP concept is similar to how the high-level AI model hides complex internal operations?
3
Inheritance
Polymorphism
Encapsulation
Overloading
Overloading
The high-level model hiding complex internal operations is similar to encapsulation in OOP.
The high-level model hiding complex internal operations is similar to encapsulation in OOP.
11
In the code example, what does the 'embedding_dim' parameter represent?
3
The size of the input vocabulary
The number of encoder layers
The dimension of word embeddings
The batch size for training
The batch size for training
The 'embedding_dim' parameter represents the dimension of word embeddings, determining how words are represented numerically.
The 'embedding_dim' parameter represents the dimension of word embeddings, determining how words are represented numerically.
12
What is the purpose of the 'vocab_size' parameter in the LanguageModel class?
2
To set the number of layers
To determine the size of the input vocabulary
To set the embedding dimension
To set the batch size
To set the batch size
The 'vocab_size' parameter determines the size of the input vocabulary, affecting the input and output layer sizes.
The 'vocab_size' parameter determines the size of the input vocabulary, affecting the input and output layer sizes.
13
Which method in the LanguageModel class is responsible for the forward pass of data?
2
init
call
fit
predict
The 'call' method in the LanguageModel class is responsible for the forward pass of data through the model.
The 'call' method in the LanguageModel class is responsible for the forward pass of data through the model.
14
What type of layer is used for the final output in the LanguageModel class?
3
LSTM
Embedding
Dense
Flatten
A Dense layer is used for the final output in the LanguageModel class.
A Dense layer is used for the final output in the LanguageModel class.
15
How many encoder layers are created in the example LanguageModel initialization?
3
1
2
3
4
The example initializes the LanguageModel with 3 encoder layers (num_layers = 3).
The example initializes the LanguageModel with 3 encoder layers (num_layers = 3).
16
What is the purpose of the 'return_sequences=True' parameter in the encoder LSTM layers?
3
To return the model to initial state
To allow sequences to be input
To output sequences for the next layer
To reverse the input sequence
To reverse the input sequence
The 'return_sequences=True' parameter allows the LSTM layer to output sequences for the next layer, maintaining temporal information.
The 'return_sequences=True' parameter allows the LSTM layer to output sequences for the next layer, maintaining temporal information.
17
Which TensorFlow module is primarily used in the code example?
2
tf.keras
tf.train
tf.estimator
tf.estimator
The code example primarily uses the tf.keras module for building the neural network model.
The code example primarily uses the tf.keras module for building the neural network model.
18
What would be a suitable method name for preprocessing input text in the LanguageModel class?
3
process_input
encode_text
preprocess_text
tokenize_input
tokenize_input
A suitable method name for preprocessing input text would be 'preprocess_text', clearly indicating its purpose.
A suitable method name for preprocessing input text would be 'preprocess_text', clearly indicating its purpose.
19
In the exercise, what functionality is suggested to be added to the LanguageModel class?
3
A method to visualize the model
A method to save the model
A method to generate text given a starting prompt
A method to evaluate the model's performance
A method to evaluate the model's performance
The exercise suggests adding a method to generate text given a starting prompt to the LanguageModel class.
The exercise suggests adding a method to generate text given a starting prompt to the LanguageModel class.
20
How does the conceptual model of AI language models help in understanding the architecture?
1
It provides a visual metaphor similar to Java objects
It explains the mathematical formulas used
It shows the exact TensorFlow implementation
It provides the training data required
It provides the training data required
The conceptual model helps understand AI language model architecture by providing a visual metaphor similar to Java objects, making it easier to grasp for those familiar with OOP concepts.
The conceptual model helps understand AI language model architecture by providing a visual metaphor similar to Java objects, making it easier to grasp for those familiar with OOP concepts.
There are no rows in this table



JULY 30:
megaphone

Describe the key components of the AI application development methodology that we covered in this class, including the use of tools like Trello, Overleaf, and synthetic personalities created with AI assistants. Explain how these elements work together to support the successful delivery of the class project.

The Development of an AI application involves several stages; the mentioned tools help to facilitate the process in every stage. For instance, for project planning:
- A Trello board helps us to manage the project. We can create a list of tasks such as "To do," "In Progress," and "Done."
- Overleaf: This tool helps us create project documentation collaboratively, such as architecture design, research papers, model selection rationale, etc.
- Synthetic personalities: Can help in the data collection, such as synthetic data generation

Train your thinking to lean into / work with the AI agents and personalities you will build to amplify your cognitive scope and bankwidth. In addition to just to answering questions: Ask your AI to advise you on concerns which you SHOULD be thinking about which you have not get thought about.
Exam Question: How does your AI KNOW what you need that you have not yet considered?

ok

Discuss the importance of the video component in the class project, as emphasized by Peter Sigurdson. Explain how creating and presenting a video can help reinforce learning and develop valuable skills for future job interviews and presentations.


You build a short look reward cycle of speaking and presenting knowledgably about these topics and getting a dopamine rush of confidence and encourage which spills over into your other arena.
image.png

Provide a comprehensive overview of the process for building the AI application that you are required to develop for the AML 3304 class, as described in our lectures. Include the following key elements: The software development methodology and tools used, such as the Unified Model Engineering Process, Trello for project management, and Overleaf for the LaTeX presentation document.

The steps involved in constructing the AI model, including the use of Google Colab notebooks, PyTorch, TensorFlow, and potentially deploying the model to platforms like Hugging Face Spaces. The importance of incorporating theoretical concepts and research questions into the project report to demonstrate an understanding of the underlying principles of AI and language models. The role of synthetic personalities created using tools like Claude AI to assist in the development and delivery of the project, including providing guidance and feedback to the students.

Mid Term Review Test Prep:

info

What are the differences between old-school methodologies like Waterfall and Agile compared with Unified Process software development processes? Discuss this in the context of the Software Crisis of the 1970s.


Modern software development methodologies put the User at the Center by gather user stories which is what Unified process does.
Modern software development methodologies are “agile” meaning, we update the plan continually as we acquire new knowledge:
<What kind of knowledge?>
Knowledge of how the Business Domain operates
Knowledge of how our development technology stack tools work: programming languages, database.



What are you giving to OPEN AI in exchange for “free” access?
Training Data.
The analysts say that the big problem with the emerging Chat GPT 5 is: There is not enough data in the world to train next Gen AI?
Synthetic Data: Perry the Stochastic Parrot
The problem with Synthetic Data? “The 2 Irish men problem”

July 12 Attendance Quiz Start of class : AI/ML engineering practices
Feature Engineering Teaching Question: Feature engineering is the process of creating new input features or transforming existing ones to improve model performance.
One common technique is binning, where continuous variables are grouped into discrete categories.
For example, age could be binned into "child", "adult", and "senior".

Q: What is the main purpose of feature binning in machine learning?
A: To convert continuous variables into categorical ones, potentially uncovering non-linear relationships and reducing noise.

CI/CD with GitHub Teaching Question:
Continuous Integration and Continuous Deployment (CI/CD) automate the building, testing, and deployment of code changes.
GitHub Actions allows you to create workflows that automatically run when certain events occur in your repository, like a push or pull request.

In a GitHub Actions workflow, what file extension is used for the configuration file? A: .yml or .yaml
(Yet Another Markup Language)

AI Model Architecture Teaching Question: The architecture of an AI model refers to its structure and components.
In a transformer model, a key component is the self-attention mechanism, which allows the model to weigh the importance of different parts of the input when processing each element.
What is the primary advantage of self-attention in transformer models?
A: It allows the model to consider the context and relationships between all parts of the input sequence.

AI Application Building with Google Colab and Hugging Face Spaces Teaching Question:
Google Colab provides a free Jupyter notebook environment with GPU support, ideal for developing AI models.
Hugging Face Spaces allows easy deployment [Model Serving environment] of these models as web applications. You create Web Route URLs
You can develop a model in Colab, save it, and then upload it to Hugging Face for deployment.

Q: What is a key advantage of using Google Colab for AI model development?
A: Free access to GPU resources for faster training and execution of models.

Transfer Learning in AI Teaching Question:
Transfer learning is a technique where a model trained on one task is repurposed on a second related task.
This approach is particularly useful when you have limited labeled (by features which is your category names) data for your target task.
For example, you might use a model pre-trained on ImageNet to classify a specific type of medical image.

Q: What is the primary benefit of using transfer learning in AI model development?
A: It allows for effective training on smaller datasets by leveraging knowledge from models trained on larger datasets. (You get the benefit of tokens, weightings, and training which someone else paid for).

June 21 Start of Class Attendance Questions:
info

Question 1

What are the key differences between deterministic algorithms and probabilistic algorithms in AI model development?
Expected Answer:
Deterministic algorithms produce the same output given the same input, with predictable and repeatable behavior. Examples include traditional rule-based systems and most classical algorithms.
Probabilistic algorithms incorporate randomness and produce different outcomes given the same input. They model uncertainty and variability in data, making them suitable for tasks like classification, prediction, and decision-making under uncertainty.

Question 2

Want to print your doc?
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
CtrlP
) instead.