Veranstaltung |
Wochentag |
Termin |
Ort |

042527 (Übung: 042528) | Montag (Übung: Mittwoch) | 10.15 - 12.00 | Otto-Hahn-Str. 12 - Raum 1.056, Campus Nord |

- Module description: pdf
- Language: English
- Lecturer: Dr. Sangkyun Lee
- TA: Sibylle Hess
- Attendance to Übung is REQUIRED.
- Office Hour (lecturer): Thursday 13:00-14:00, LS8, OH 12, Raum 4.023 (or by appointments)

04.03.2016 (Friday) 10:00-12:00, OH12 R4.013

- Use this form to register for the exam pdf.
- Exam questions will be from lecture notes, discussions in class, and homeworks including bonus questions.
- You can bring an A4 paper with your own summary note to the exam, but using ONLY ONE SIDE of the paper.

In this lecture we will learn theories and algorithms of numerical optimization. We study the mathematical structure of various optimization problems to design efficient algorithms. The structure is investigated by accessing the zero-th order (function values), the first order (derivatives), and the second order information (Hessians) about the objective function, as well as by looking into the geometry of constraints. We discuss constrained and unconstrained optimization problems in continuous spaces, focusing on understanding motivations behind technical details and analyzing convergence rate / algorithm complexity algorithms. Fundamental concepts such as optimality and duality will be discussed in detail, which also become popular tools to better understand algorithms in many areas including machine learning, data mining, and statistics. The importance of smoothness and convexity will be elaborated, especially in connection to regularization problems in high dimensions. Some advanced topics from non-smooth, large-scale, or matrix optimization will be included if time permits. Homework assignments will be given to check theoretical and practical understanding of techniques.

The aim of this lecture is to provide students with understanding of fundamental concepts and techniques in optimization in an advanced level, so that students can understand and see how to use and design efficient numerical optimization algorithms for their own research problems.

- Numerical Optimization, J. Nocedal and S. Wright, 2nd Ed, Springer, 2006
- Introductory Lectures on Convex Optimization, Y. Nesterov, Springer, 2004
- Nonlinear Programming, D. P. Bertsekas, 2nd Ed., Athena Scientific, 2003 (2nd printing)
- Convex Optimization, S. Boyd and L. Vandenberghe, Cambridge, 2004 [pdf]

- (19.10) Lecture 1: Introduction
- > (optional reading) Intro to compressed sensing pdf

- A homework will be given approx. every two weeks. Some will require implementing algorithms in R or Matlab.
- Exercises are at the end of each chapter in the lecture notes.
__Submit__your handwritten (must be readable)or TEX-typesetted answer at the__BEGINNING of the lecture__on the due date. On emergency, you can submit by email to the lecturer, before the lecture begins.- Dicussion with other students are encouraged, but
__copying is STRICTLY prohibited__(you will not be allowed for the final exam). A rule of thumb: never share your written solutions to other studnets.

- > Homework 1 (due 2.11, beginning of the class): exercises 1.1, 1.2 (a), 1.3, 1.4, and 1.5 (1.1 has been updated on 21.11, so please check the recent version)
- > Homework 2 (due 16.11, beginning of the class): exercises 2.5, 3.1, 3.2, 4.1, **4.3, 4.4. (numbers marked with ** are optional for extra points)
- > Homework 3 (due 30.11, 10:15am. Submit by email to the TA, Sibylle): exercises 5.2[10], 5.3[10], 6.1[10], 6.2 (1)[5], (2)[20], (3.a)[10], (3.b)[5], **5.1[10], **6.2(3.c)[5] (6.2 requres Matlab/Octave coding). Numbers in square brackets are the scores. Submit your latex typesetted solution, program code, a script to demo your code (zip everthing into a single achieve, firstname.lastname.zip or .tar.gz) MNIST data set: download
- > Homework 4 (due 11.01.16 10:15am. Submit by email to the TA, Sibylle): exercise 8.1 (requires Matlab/Octave coding). Submit your latex typesetted solution, program code, a script to demo your code (zip everthing into a single achieve, firstname.lastname.zip or .tar.gz)
- > Homework 5 (due 25.01.16 10:15am. Submit by email to the TA, Sibylle): exercise 9.1, 9.2(a) and *9.2(b). 9.2(a) requires coding, and 9.2(b) is optional for bonus points. Submit your latex typesetted solution, program code, a script to demo your code (zip everthing into a single achieve, firstname.lastname.zip or .tar.gz)
- > Homework 6 (due 8.2.16. Submit in the beginning of the class at 10:15am): exercise 10.1, 10.3, 10.5, 10.6, 11.1, and *11.2. Problems marked with * are for bonus points.