Τμήμα Ηλεκτρολόγων Μηχανικών & Μηχανικών Υπολογιστών - Dept. of Electrical & Computer Engineering


Πανεπιστήμιο Θεσσαλίας (Βόλος) - University of Thessaly (Volos)

Disclaimer

The present Web page is not a substitute for the class; it will be updated asynchronously and incompletely

Summary

  1. The course will deal with:
    Basic concepts in Neural Networks (NNs) design and operation, and elements of Fuzzy Subset Theory
  2. The course focuses on the following types of NNs:
    Feedforward NNs: MultiLayer Perceptrons (MLPs), Convolutional NNs (CNNs)
    Radial Basis Function NNs
    Competitive NNs: Self-Organizing Map (SOM), Learning Vector Quantization (LVQ)
    Hebbian NNs: Linear Autoencoder
    Recurrent NNs: Layered Digital Dynamic Networks (LDDN) in general, Long-Short Term Memory (LSTM), Gated Recurrent Units (GRUs)
    Optimization methods for training deep neural networks: (Stochastic) Gradient Descent, Adagrad, Adadelta, Adam, Newton, Conjugate Gradient, BFGS
    Deep architectures of NNs: MLPs, CNNs, RNNs, Transformers
    Distributed learning: (Purely Distributed, Hybrid) Federated Deep Learning
    Fuzzy Subset Theory and Fuzzy Logic
  3. The course aims to teach the use of the aforementioned in:
    Computations (e.g., in function approximation, regression, classification, clustering)

Instructor

Δημήτριος Κατσαρός (prof. Dimitrios Katsaros)

Recommended literature

Book
Title Βαθιά Μάθηση
Local ΕΝ copy 		Neural Network Design
Useful material 		Dive into Deep Learning
Useful site
Authors Ian Goodfellow et al. Marin Hagan et al. Aston Zhang et al.
Language Greek English English

Data Science and Deep Learning: A perspective

The instructor's opinion on What is Data Science and Deep Learning.

Assignments and Grading:

Lectures will take place in Room 111

Πέμπτη (Thursday) 11:00-13:00 και (and) Παρασκευή (Friday) 11:00-13:00

Ασκήσεις αυτο-αξιολόγησης (Self-evaluation exercices)

Release: 24 Sept. 2024. No return
Η εκφώνηση βρίσκεται εδώ.

Σειρά-προβλημάτων-01 (Problem-Set-01)

Release: Nov. 04, 2024, Deadline: Dec. 01, 2024
Η εκφώνηση βρίσκεται εδώ.
You need to study: Backpropagation for Convolutional Neural Networks.

You may find here a Summary of Lectures (Σύνοψη των διαλέξεων)

Πρόγραμμα διαλέξεων (Lectures schedule)

Εβδομάδα
(Week)		 Ημερομηνία
(Date)		 Αντικείμενο διάλεξης
(Topic)		 Διαφάνειες (Slides)
(1st part)		 Διαφάνειες (Slides)
(2nd part)		 Σύνδεσμοι στις σχετικές εργασίες
(Links to papers)
1 26-27/09/2024 α) Introduction to Neural Networks
β) Introduction to TensorFlow 		Διάλεξη 1 Διάλεξη 2 link-1
2 03-04/10/2024 α) The basic Perceptron architecture and learning algorithm
β) Activation functions 		Διάλεξη 3 Διάλεξη 4 link-2
link-3
3 10-11/10/2024 α) The ADALINE (Widrow-Hoff) neural network
β) Multi-layer Perceptron and the backpropagation algorithm 		Διάλεξη 5 Διάλεξη 6 link-4
link-5
link-6
4 24-25/10/2024 α) Exercices on ADALINE and Backpropagation
 Διάλεξη 7 Διάλεξη 8
5 31/10-01/11/2024 α) Heuristic variations on backpropagation: Momentum, Variable learning rate
β) Optimization-based backpropagation: Conjugate gradient, Levenberg-Marquardt, Adagrad, RMSProp, Adadelta, Adam, AdaHessian 		Διάλεξη 9

Διάλεξη 9 Συμπληρωματική: Dropout

Διάλεξη 9 Συμπληρωματική: Critique of activation functions

Διάλεξη 9 Συμπληρωματική: Regularization techniques

Stability of SD with momentum
 Διάλεξη 10 link-7
link-8
link-9
link-10
link-11
link-12
link-13
link-14
link-15
link-16

Optimizers for deep models

link-17
link-18
link-19
link-20
link-21
link-22
link-Benchmarking_DL_optimizers
ADOPT: champion adaptive gradient
6 07-08/11/2024 α) Convolutional neural networks Ι
β) Convolutional neural networks ΙI 		Διάλεξη 11 Διάλεξη 12
Διάλεξη 12 Συμπληρωματική: Batch Normalization layer 		link-23
link-24
link-25
7 14-15/11/2024 α) Backpropagation for CNNs
β) Exercises on CNNs 		Διάλεξη 13
8 21-22/11/2024 α) Radial-Basis Function Neural Networks
β) Training RBF networks with Linear Least Squares (LLS)
γ) Hamming network and Competitive Learning 		Διάλεξη 14
Διάλεξη 15 		Διάλεξη 16
Διάλεξη 17 		link-26
link-27
link-28
link-29
link-30
link-31
9 28-29/11/2024 α) Self-Organizing Feature Map and Learning Vector Quantization
β) Exercises on Kohonen's rule and on SDBP with momentum 		Διάλεξη 18 Διάλεξη 19 link-32
10 05/12/2024 α) Supervised Hebbian learning
β) Exercises 		Διάλεξη 20 link-33
11 12-13/12/2024 α) (Dynamic) Recurrent networks: BackProp for two- and three-layer RNNs
β1) Dynamic networks: Real Time Recurrent Learning (RTRL)
β2) Dynamic networks: Backpropagation Through Time (BPTT)
γ) Modern RNNs: LSTM, GRU 		Διάλεξη 21 Διάλεξη 22
Διάλεξη 23
Intro to LLMs
GPT-4 Technical Report (no info: arch, training, dataset)
Course to get into LLMs
 link-34
link-35
link-36
link-37
link-38
link-39
link-40
12 19-20/12/2024 α) Transformers
β) Federated learning 		Διάλεξη 24 Διάλεξη 25 link-41
link-42
link-43
13 09-10/01/2025 (θα μεταφερθεί εντός του 2024) Fuzzy Subset Theory and Fuzzy Logic Διάλεξη 26 Intro to Fuzzy logic

dkatsar AT uth DOT gr
Τελευταία ενημέρωση: Δευ. 18 Νοεμβρίου 2024