A Simple Way to Optimize “Something” in Python

Solving budget optimization problems using linear programming in Python

Why is optimizing a budget important? It allows us to control ads spending, determine how much to spend and maximize desired outcome (visits, clicks etc). As an example, given a budget of $10,000 and its constraints, we are able to determine what is the optimal budget allocation for each marketing channel such that we maximize the outcome. It would allow us to avoid crowding on a single channel where we put everything into a single basket but spreading it out to best performing channels.

A quality budget optimization strategy can significantly improve the effectiveness of search advertising campaigns, thus helping advertisers to succeed in the fierce competition of online marketing.

What is linear programming?

Graphical solution of a linear programming problem by Mark Schulze in Linear Programming Optimization, 2005

Linear programming (LP) is one of the most widely used optimization technique to obtain the best outcome. An excerpt from the book “Linear and Nonlinear Programming” by David G. Luenberger:

A linear programming problem is characterized, as the name implies, by linear functions of the unknowns; the objective is linear in the unknowns, and the constraints are linear equalities or linear inequalities in the unknowns.

It seems that the popularity of linear programming lies primarily with the formulation phase of analysis rather than the solution phase — and for good cause. Thus, for the problem that we are about today, it can be formulated as:

A budget constraint restricting the total amount of money to be allocated among three different channels (TV, Radio, Newspaper), the budget constraint takes the form x1 + x2 + x3 ≤ B, where B is the budget. The total spend for TV should be less than $200, Radio should be less than $500 and Newspaper should be less than $500. While total budget is capped at $1000.

Aside from that, we should also find out the objective function for a problem where we plan to maximize or minimize our preferred outcome (click, sales etc).

Sample problem formulation for linear programming from the book Linear and Non-linear programming by David G. Luenberger, 1973

I’ll be using an open-source advertising dataset from Kaggle to show you how it works!

Implementation

Ideally, when working with LP, the objective function should be linear by virtue.

If an objective function is not purely linear by virtue of its inherent definition, it is often far easier to define it as being linear than to decide on some other functional form and convince others that the more complex form is the best possible choice. Linearity, therefore, by virtue of its simplicity, often is selected as the easy way out or, when seeking generality, as the only functional form that will be equally applicable (or nonapplicable) in a class of similar problems — David G.Luenberger

The Optimization Process by PuLP, 2009

Build a prediction model

Dataframe

We start off by building a multivariate linear regression model using SK-Learn’s linear regression. This kernel provided a great way to validate whether linear regression can be used as linear regression is bounded by several assumptions.

### SCIKIT-LEARN ###

feature_cols = ['TV', 'Radio', 'Newspaper']
X = df_advertising[feature_cols]
y = df_advertising[["Sales"]]

# instantiate and fit
lr = LinearRegression()
model = lr.fit(X, y)

# print the coefficients
print(SkLearn_result.intercept_)
print(SkLearn_result.coef_)

Y-intercept and coefficients

We are able to obtain the y-intercept and coefficients from our given model.

Accuracy of our model

With a RMSE of 1.509, our model is pretty accurate and based on the diagram above, we are able to see that it can predict relatively well as compared to the actual value. With the prediction model in place, we can now move towards building our objective function for linear programming.

Optimizing using LP

We will be using a python packaged called PuLP. It is a optimization package for Python. PuLP is straight-forward and very easy to be used!

prob = LpProblem("Ads Sales Problem", LpMaximize)

We start off by defining the problem using LpProblem function where we wanted to maximize the output thus “LpMaximize” would be the parameter. Then we would specify our constraints such as how much we budget should be spent for a given channel.

#TV <= 200
x = LpVariable("x", 0, 200)

#Radio <= 500
y = LpVariable("y", 0, 500)

#Newspaper <= 500
z = LpVariable("z", 0, 500)

#Should be less than $1000
prob += x + y + z <= 1000

With the constraints in place, we can now build our objective function using the coefficients given by the model:

coef = model.coef_

prob += coef.item(0) * x + coef.item(1) * y + coef.item(2) * z + model.intercept_[0]

Then by calling the solve() function, it will solve it for us and we can also check the status of our optimization. By printing it, we can obtain the below result.

prob.solve()
LpStatus[status]

How do we know the optimal value for each variable?

Just using a simple for-loop to access the prob.variables, we are able to obtain the optimal value for each variable and also the maximum objective value below.

for v in prob.variables():
print(v.name, "=", v.varValue)

print("Objective = %f" % (prob.objective.value()))

Analysis

Now we know how much should we spend per marketing channel, we should definitely use it as a baseline model to allocate our budgets for our next marketing campaign. We can actually automate the entire process where the budget can be allocated or optimized throughout each day for a single campaign.

Conclusion

Linear programming for optimization is powerful when it can be applied to the right use-case as it provides key managerial insights. Advertisers usually take the budget as simple constraints and put a lot of efforts to find more effective ways for possible operations as defined by various kinds of markets. A simple strategy for budget allocation and adjustment can significantly minimize the loss in terms of effective clicks/revenue/sales in campaign management.

Reference

Linear and Non-linear programming https://www.springer.com/gp/book/9780387745022
PuLP — http://coin-or.github.io/pulp/
Repository for this tutorial — https://github.com/georgeblu1/Data-Projects/blob/master/Budget%20Optimization.ipynb
https://towardsdatascience.com/linear-programming-and-discrete-optimization-with-python-using-pulp-449f3c5f6e99

Citation for image

Feel free to reach out to me if you have any questions or feedback for me through my LinkedIn. Quick introduction, I am a data scientist at Airasia.com, focused on building great products and driving growth because I just love doing what I do.