Advanced Topics in Artificial Intelligence
NEWS
mar 20: - slides from 19/3 in page - text on Gibbs & VE revised.
COVID-19
Dear Students.
Due to the current limitations, classes are being given through Zoom video-conference, and:
-
i will be available to answer questions through Zoom at the following times: + Mon, 11:00am--11:00am + Wed, 10:00am--11:00am + Thus, 9:30
-
The test date is moved to the exam
-
MiniProjects schedules are maintained
-
I will be revising the page and adding complementary information.
Take care and thanks!
Vítor
Program
-
Graphical Systems (Probabilistic)
-
Neural Systems
-
Logic Based Aproaches
-
Other representations
-
Integrated Systems
Instructors
-
Email: vsc@dcc.fc.up.pt
Classes
Class I - Welcome Class [14/02]
- Enjoying the AI Summer: Introductory Slides
Class II - The beginning is a good place to start [14/02]
- Concepts in AI: [David Page/MarkCraven](David Page/MarkCraven plus methodology
Class III - BFF: Linear Regression and AI [21/02]
- Concepts in AI: David Page/MarkCraven
Class IV - Seeing Things: Linear Regression and AI [21/02]
- Ridge regression
- Lasso Regression
- Experimental Evaluation
- Probability
Class V - A Light in Darkness: Probabilistic Graphical Models [28/02]
- Generative, directed models
- Naive Bayes
- Parameter as optimisers
- Inference
Slides available for Naive Bayes.
Class V - Into the Deeps: Deep Learning Basics
-
- Variable Elimination and Gibbs sampling:
- Introduce to deep networks: original slides based on chapter VI of Deep Learning books; also used material from [G Hinton's Coursera lectures[(https://www.deeplearningbook.org/)
Mini-Tasks
Inference
Implement a program to output the posterior probabilities of a random variable given a bayes network and evidence.
-
The R bnlearn Repo has a collection of Bayes nets that you can use.
-
Implement the algorithm you choose: look into this text for a brief overview of two simple algrithms: Gibbs Sampling and Variable Elimination. Feel free to use existing BN scanner. Last, implement a small query answering mechanism.
-
Evaluate by comparing with existing libraries.
Eval criteria:
- 20% on presentation: readability, completeness, structure.
-
20% on getting and pre-processing the datasets
-
30% on running the algorithm correctly.
-
30% on performance (ie, how well you used the DNNs) and difficulty:
-
how many datasets can you run?
Learning from data
The goal is to compare deep network technology on a dataset: what kind of techniques will be useful, how does searhcing ?
-
First, you should choose a dataset
-
Is it large enough? Too Large? Too Many Parameters? Too few? Missing Data?
-
Are there published results? If so, are they reproducible?
-
Try Naive Bayes(or linear regression) and RFs: they will be the baseline.
-
Generate a DNN. Ways to do so, many are. A way is by starting from the keras tutorials in tf.org.
-
Refine the model.
-
Evaluate on set-aside data, or using cross-val.
eval criteria:
-
20% on presentation: readability, completeness, structure.
-
20% on getting and pre-processing the datasets
-
30% on workable DNN models
-
30% on performance (ie, how well you used the DNNs) and difficulty.
Deadlines:
- Inference: Easter
- DNN Datasets: Easter
Submission:
To submit you must present - source + small report + run log (eg, jupyter notebook) to be sent to vscosta AT fc.up.pt, subj TAIA.
Support Material
Tutorials
Why attend a tutorial? Introduce, explain and comment on the material on n ai tutorial. Look for good tutorials at:
- IJCAI
- AAAI
- ECAI
- ICML
- NIPS
- ECML
- KDD
- ICDM
Slides
Probabilistic Systems
Main slides, also check quick reading slides:
-
Adrian Weller, MLSALT4 graphical model
-
Marc Toussaint University of Stuttgart Summer 2015, Machine Learning, Graphical Models
-
For detailed information, try Daphne Koller Open Class Slides
Not-so-quick Reading:
a. Daphne Koller and Nir Friedman, Probabilistic Graphical Models: everything you wanted and everything you didn't
b. Kevin P. Murphy, Machine Learning A Probabilistic Learning: A probabilistic view of the world.
c. David Barber, Bayesian Reasoning and Machine Learning: pdf from author available/ <!--
Propositional Inference
Following slides discuss the connection between SAT solvers, BDDs and trees:
-
Binary Decision Diagrams are one of the most widely used tools in CS. Their application to BN was proposed by Minato et al but they are not a very popular approach to compile BNs. Model Counting is also not widely used.
-
The Problog language was initially implemented on BDDs. The ProbLog2 system can use BDDs. model-counting or trees.
-
Arithmetic Circuits are a very efficient approach for bayesian inference; the Darwiche group recently proposed an extension, SDDs.
-
The connection to constraint solving is discussed by Dechter.
Optimisation
Optimisation is a fundamental tool for modern machine learning and other areas of AI. Techniques are often based on work from the Operations Research and Constraint communities.
- Discrete domains are used in areas such as planning and game playing. They usually alternate between search and propagation, and are often implemstart.