Advanced Topics in Artificial Intelligence
NEWS
mar 20:  slides from 19/3 in page  text on Gibbs & VE revised.
COVID19
Dear Students.
Due to the current limitations, classes are being given through Zoom videoconference, and:

i will be available to answer questions through Zoom at the following times: + Mon, 11:00am11:00am + Wed, 10:00am11:00am + Thus, 9:30

The test date is moved to the exam

MiniProjects schedules are maintained

I will be revising the page and adding complementary information.
Take care and thanks!
Vítor
Program

Graphical Systems (Probabilistic)

Neural Systems

Logic Based Aproaches

Other representations

Integrated Systems
Instructors

Email: vsc@dcc.fc.up.pt
Classes
Class I  Welcome Class [14/02]
 Enjoying the AI Summer: Introductory Slides
Class II  The beginning is a good place to start [14/02]
 Concepts in AI: [David Page/MarkCraven](David Page/MarkCraven plus methodology
Class III  BFF: Linear Regression and AI [21/02]
 Concepts in AI: David Page/MarkCraven
Class IV  Seeing Things: Linear Regression and AI [21/02]
 Ridge regression
 Lasso Regression
 Experimental Evaluation
 Probability
Class V  A Light in Darkness: Probabilistic Graphical Models [28/02]
 Generative, directed models
 Naive Bayes
 Parameter as optimisers
 Inference
Slides available for Naive Bayes.
Class V  Into the Deeps: Deep Learning Basics

 Variable Elimination and Gibbs sampling:
 Introduce to deep networks: original slides based on chapter VI of Deep Learning books; also used material from [G Hinton's Coursera lectures[(https://www.deeplearningbook.org/)
MiniTasks
Inference
Implement a program to output the posterior probabilities of a random variable given a bayes network and evidence.

The R bnlearn Repo has a collection of Bayes nets that you can use.

Implement the algorithm you choose: look into this text for a brief overview of two simple algrithms: Gibbs Sampling and Variable Elimination. Feel free to use existing BN scanner. Last, implement a small query answering mechanism.

Evaluate by comparing with existing libraries.
Eval criteria:
 20% on presentation: readability, completeness, structure.

20% on getting and preprocessing the datasets

30% on running the algorithm correctly.

30% on performance (ie, how well you used the DNNs) and difficulty:

how many datasets can you run?
Learning from data
The goal is to compare deep network technology on a dataset: what kind of techniques will be useful, how does searhcing ?

First, you should choose a dataset

Is it large enough? Too Large? Too Many Parameters? Too few? Missing Data?

Are there published results? If so, are they reproducible?

Try Naive Bayes(or linear regression) and RFs: they will be the baseline.

Generate a DNN. Ways to do so, many are. A way is by starting from the keras tutorials in tf.org.

Refine the model.

Evaluate on setaside data, or using crossval.
eval criteria:

20% on presentation: readability, completeness, structure.

20% on getting and preprocessing the datasets

30% on workable DNN models

30% on performance (ie, how well you used the DNNs) and difficulty.
Deadlines:
 Inference: Easter
 DNN Datasets: Easter
Submission:
To submit you must present  source + small report + run log (eg, jupyter notebook) to be sent to vscosta AT fc.up.pt, subj TAIA.
Support Material
Tutorials
Why attend a tutorial? Introduce, explain and comment on the material on n ai tutorial. Look for good tutorials at:
 IJCAI
 AAAI
 ECAI
 ICML
 NIPS
 ECML
 KDD
 ICDM
Slides
Probabilistic Systems
Main slides, also check quick reading slides:

Adrian Weller, MLSALT4 graphical model

Marc Toussaint University of Stuttgart Summer 2015, Machine Learning, Graphical Models

For detailed information, try Daphne Koller Open Class Slides
Notsoquick Reading:
a. Daphne Koller and Nir Friedman, Probabilistic Graphical Models: everything you wanted and everything you didn't
b. Kevin P. Murphy, Machine Learning A Probabilistic Learning: A probabilistic view of the world.
c. David Barber, Bayesian Reasoning and Machine Learning: pdf from author available/ <!
Propositional Inference
Following slides discuss the connection between SAT solvers, BDDs and trees:

Binary Decision Diagrams are one of the most widely used tools in CS. Their application to BN was proposed by Minato et al but they are not a very popular approach to compile BNs. Model Counting is also not widely used.

The Problog language was initially implemented on BDDs. The ProbLog2 system can use BDDs. modelcounting or trees.

Arithmetic Circuits are a very efficient approach for bayesian inference; the Darwiche group recently proposed an extension, SDDs.

The connection to constraint solving is discussed by Dechter.
Optimisation
Optimisation is a fundamental tool for modern machine learning and other areas of AI. Techniques are often based on work from the Operations Research and Constraint communities.
 Discrete domains are used in areas such as planning and game playing. They usually alternate between search and propagation, and are often implemstart.