## Advanced Topics in Artificial Intelligence

### NEWS

mar 20: - slides from 19/3 in page - text on Gibbs & VE revised.

### COVID-19

Dear Students.

Due to the current limitations, classes are being given through Zoom video-conference, and:

• i will be available to answer questions through Zoom at the following times: + Mon, 11:00am--11:00am + Wed, 10:00am--11:00am + Thus, 9:30

• The test date is moved to the exam

• MiniProjects schedules are maintained

• I will be revising the page and adding complementary information.

Take care and thanks!

Vítor

### Program

1. Graphical Systems (Probabilistic)

2. Neural Systems

3. Logic Based Aproaches

4. Other representations

5. Integrated Systems

# Classes

#### Class IV - Seeing Things: Linear Regression and AI [21/02]

• Ridge regression
• Lasso Regression
• Experimental Evaluation
• Probability

#### Class V - A Light in Darkness: Probabilistic Graphical Models [28/02]

• Generative, directed models
• Naive Bayes
• Parameter as optimisers
• Inference

Slides available for Naive Bayes.

#### Class V - Into the Deeps: Deep Learning Basics

• Variable Elimination and Gibbs sampling:
• Introduce to deep networks: original slides based on chapter VI of Deep Learning books; also used material from [G Hinton's Coursera lectures[(https://www.deeplearningbook.org/)

#### Inference

Implement a program to output the posterior probabilities of a random variable given a bayes network and evidence.

1. The R bnlearn Repo has a collection of Bayes nets that you can use.

2. Implement the algorithm you choose: look into this text for a brief overview of two simple algrithms: Gibbs Sampling and Variable Elimination. Feel free to use existing BN scanner. Last, implement a small query answering mechanism.

3. Evaluate by comparing with existing libraries.

##### Eval criteria:
• 20% on presentation: readability, completeness, structure.
• 20% on getting and pre-processing the datasets

• 30% on running the algorithm correctly.

• 30% on performance (ie, how well you used the DNNs) and difficulty:

• how many datasets can you run?

#### Learning from data

The goal is to compare deep network technology on a dataset: what kind of techniques will be useful, how does searhcing ?

1. First, you should choose a dataset

2. Is it large enough? Too Large? Too Many Parameters? Too few? Missing Data?

3. Are there published results? If so, are they reproducible?

4. Try Naive Bayes(or linear regression) and RFs: they will be the baseline.

5. Generate a DNN. Ways to do so, many are. A way is by starting from the keras tutorials in tf.org.

6. Refine the model.

7. Evaluate on set-aside data, or using cross-val.

##### eval criteria:
• 20% on presentation: readability, completeness, structure.

• 20% on getting and pre-processing the datasets

• 30% on workable DNN models

• 30% on performance (ie, how well you used the DNNs) and difficulty.

• Inference: Easter
• DNN Datasets: Easter

#### Submission:

To submit you must present - source + small report + run log (eg, jupyter notebook) to be sent to vscosta AT fc.up.pt, subj TAIA.

## Support Material

### Tutorials

Why attend a tutorial? Introduce, explain and comment on the material on n ai tutorial. Look for good tutorials at:

• IJCAI
• AAAI
• ECAI
• ICML
• NIPS
• ECML
• KDD
• ICDM

# Slides

### Probabilistic Systems

Main slides, also check quick reading slides:

1. Adrian Weller, MLSALT4 graphical model

2. Marc Toussaint University of Stuttgart Summer 2015, Machine Learning, Graphical Models

3. For detailed information, try Daphne Koller Open Class Slides

a. Daphne Koller and Nir Friedman, Probabilistic Graphical Models: everything you wanted and everything you didn't

b. Kevin P. Murphy, Machine Learning A Probabilistic Learning: A probabilistic view of the world.

c. David Barber, Bayesian Reasoning and Machine Learning: pdf from author available/ <!--

### Propositional Inference

Following slides discuss the connection between SAT solvers, BDDs and trees:

1. Binary Decision Diagrams are one of the most widely used tools in CS. Their application to BN was proposed by Minato et al but they are not a very popular approach to compile BNs. Model Counting is also not widely used.

2. The Problog language was initially implemented on BDDs. The ProbLog2 system can use BDDs. model-counting or trees.

3. Arithmetic Circuits are a very efficient approach for bayesian inference; the Darwiche group recently proposed an extension, SDDs.

4. The connection to constraint solving is discussed by Dechter.

### Optimisation

Optimisation is a fundamental tool for modern machine learning and other areas of AI. Techniques are often based on work from the Operations Research and Constraint communities.

• Discrete domains are used in areas such as planning and game playing. They usually alternate between search and propagation, and are often implemstart.