Optimization Modeling for Engineers

ME 5374 (special topics)
Northeastern University

Instructor: Laurent Lessard

This course provides an introduction to optimization modeling in engineering contexts. Topics covered include linear programming, quadratic and convex programs, and mixed-integer and non-convex models. By the end of the course, students will be equipped to transform practical engineering challenges into optimization models, assess their difficulty, solve them using modern software tools, interpret their solutions, and perform appropriate sensitivity and trade-off analyses. Examples will be drawn from all areas of engineering, as well as computer science and business/finance. Students should have taken a course on linear algebra (ME 2341 or equivalent) to register for this course.

This is a new class! First offering will be Spring 2024

What is “optimization modeling”?

Optimization modeling is a process, and it starts with a description of a decision-making problem. All branches of engineering involve some form of decision-making! Here are a few examples:

  • Design: Finding the best shape for a truss structure, ensuring structural integrity but also making it as light and inexpensive as possible.
  • Supply chains: Finding the most efficient routes and schedules for shipping goods from multiple warehouses to various destinations, while minimizing transportation costs and meeting delivery deadlines.
  • Manufacturing: Deciding the quantity of each product to produce in a factory to maximize profit, given constraints like production capacity, raw material availability, and market demand.
  • Energy Usage in Buildings: Adjusting heating, ventilation, and air conditioning (HVAC) systems to minimize energy consumption while maintaining comfort levels in different zones of a building.
  • Scheduling: Allocating staff in a hospital or customer service center to ensure adequate coverage at all times while minimizing labor costs and meeting service level agreements.
Such problem formulations are often vague and incomplete. The next step is to make the formulation precise: identify the decision variables, objective function, and constraints. This class will develop your intuition (mathematical and geometrical) for optimization problems so that you can answer questions such as: How difficult will it be for a computer to solve this problem? What sorts of solutions can we expect to find? The math prerequisites are linear algebra and calculus.

The final step is to solve the mathematical model using a computer and interpret the solutions. This will require some coding; you will learn to use a high-level modeling language and call open-source and commercial solvers to do the heavy lifting. This is not a coding class; we will not be implementing any algorithms ourselves, and no prior coding experience is needed. Nevertheless, we will learn the basics of different algorithms to understand their applications and what to expect when using them. Finally, we will learn to interpret the solutions and perform any relevant sensitivity analyses.

Target audience for the course

  • Undergraduate students, generally seniors in ME, ECE, CEE, etc., who are graduating soon and realize they don’t know anything about optimization and want a “one-stop-shop” that will teach them how to recognize optimization problems in the wild and gain some basic literacy on the topic. The course has a project component, which would be an opportunity for students to include an optimization component in one of their large projects (e.g., Capstone/design).
  • Graduate students that want to add breadth in optimization to complement their existing domain expertise. In the course project, these students could explore the use of optimization modeling for their current/ongoing research projects.
  • Students considering IE/OR that are unsure where to begin and don’t really know/understand what is out there. This course can serve as a “gateway” course that provides enough breadth that students gain an idea of what interests them and where they can go from there.
  • Students in Khoury (undergraduate or graduate) interested in optimization. CS courses tend to focus more on algorithms, so an optimization modeling course such as this one would complement existing algorithms courses nicely.
  • If you are enrolled in an IE/OR program and are considering taking this course to complement the other optimization courses in your program, please consult your academic advisor.

What this class is not

  • An algorithms class. We will briefly discuss the main classes of algorithms that modern solvers use so that you have a rough understanding of how different solvers work. Our discussions will cover the strengths and weaknesses of different algorithms (what sorts of guarantees do they provide? How efficient are they? How scalable are they?). Although we will be using state-of-the-art solvers in the class, we will not be coding/implementing any algorithms ourselves.
  • A coding class. Optimization problems must generally be solved using a computer, so some amount of coding is inevitable… But this is not coding in the traditional sense. We will be using a “modeling language” that makes implementation relatively straightforward. No prior coding experience is needed.
  • A machine learning class. Neural networks are generally viewed as a last resort in the context of engineering decision-making and planning problems, because they are computationally intensive and typically lead to uninterpretable solutions with few guarantees. That being said, machine learning problems can be framed as optimization models; learning about optimization can improve your understanding of machine learning. We will make these connections apparent in the class, as time permits. To learn about ML specifically, I suggest taking one of the many dedicated ML courses offered at Northeastern.
  • A numerical methods class. Numerical methods are an important aspect of engineering analysis and design. Here, I’m using “numerical methods” to mean the solution of ordinary or partial differential equations over a spatiotemporal domain. The main examples are finite element analysis (FEA) and computational fluid dynamics (CFD). While FEA and CFD can be viewed as optimization problems, they are highly specialized and you should take a dedicated course on FEA or CFD to learn about these topics.

Topics covered in the course:

You should not know what most of the following keywords mean — if you do, this class is likely not for you!

  • Linear programs. Geometrical interpretation, polytopes, assignment problems, minimax and planning problems, network flow problems, sensitivity analysis via LP duality.
  • Quadratic programs. Least squares, minimum-norm, normal equations, equality constraints, trade-offs, quadratic forms, QCQPs, geometry of solutions.
  • Conic programs. Cones, SOCPs, semidefinite programs.
  • Convex programs. KKT conditions, sensitivity analysis via Lagrangian duality.
  • Integer programs. Rounding and relaxation, fixed costs and variable bounds, logic constraints, set cover problem, traveling salesperson problem, quadratic assignment, special ordered sets.
  • Nonlinear programs. nonlinear least squares, modeling techniques.

Evaluation:

Formative (about 20%)

  • Participation: In-class discussions and problem-solving. To practice the skills you learn as you’re learning them.
  • Weekly homework: Solving modeling problems of different types as the class progresses. To practice what you learned in class and to give yourself an opportunity to gage your grasp of the material and identify gaps in knowledge.

Summative (about 80%)

  • In-class exams: (non-cumulative). To demonstrate your grasp of the course material.
  • Final class project: Optimization modeling project (work in teams) to demonstrate your ability to apply your knowledge of the course material to model and solve an optimization problem. The project can be based on a problem encountered in one of your ongoing projects (e.g. Capstone, or MS/PhD research).