{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Linear Models\n", "\n", "There's a few linear models out there that we felt were generally useful. This\n", "document will highlight some of them. \n", "\n", "## Least Absolute Deviation Regression\n", "\n", "Imagine that you have a dataset with some outliers." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA54AAAD4CAYAAACEyjk9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAT+0lEQVR4nO3df4xd6XkX8O+DvZVm28AEdvrDs1m8VMFQuolcBigs0LBp5E2IErMqUkN/hBBkIaBUiLpZq1LzR4V2kfnRotJGVrpsK6Lkj9S4QaV1V12lC20SmI03cZqt2yhptzsbWC/BBaWDsuu8/OFx1p6MPddz73tnzr2fj2R57rln5jy6eu/M/Z73Pc+p1loAAACglz+y2wUAAAAw2wRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK72T/Ngd9xxRzt48OA0DwkAAMCUPPnkky+01pY2b59q8Dx48GBWV1eneUgAAACmpKp+b6vtltoCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0NVUu9oCAMBecebcWk6evZDnLq3nwOJCjh85lKOHl3e7LJhJgicAAHPnzLm1nDh9PusvXk6SrF1az4nT55NE+IQOLLUFAGDunDx74Suh86r1Fy/n5NkLu1QRzDbBEwCAufPcpfVb2g6MR/AEAGDuHFhcuKXtwHgETwAA5s7xI4eycNu+67Yt3LYvx48c2qWKYLZpLgQAwNy52kBIV1uYDsETAIC5dPTwsqAJU2KpLQAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQ1bbBs6oeqarnq+pTWzz3Q1XVquqOPuUBAAAwdKPMeD6a5P7NG6vqVUnekOSZCdcEAADADNk2eLbWnkjyhS2e+jdJfjhJm3RRAAAAzI4dXeNZVW9JstZa+8QI+x6rqtWqWr148eJODgcAAMCA3XLwrKrbk/xIkh8dZf/W2qnW2kprbWVpaelWDwcAAMDA7WTG85uT3J3kE1X1u0nuTPLxqvrGSRYGAADAbNh/q9/QWjuf5OuvPt4InyuttRcmWBcAAAAzYpTbqbw/yUeSHKqqZ6vqnf3LAgAAYFZsO+PZWnvbNs8fnFg1AAAAzJwddbUFAACAUQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANDVtsGzqh6pquer6lPXbDtZVb9VVZ+sqv9YVYt9ywQAAGCoRpnxfDTJ/Zu2PZbkW1trr0ny20lOTLguAAAAZsS2wbO19kSSL2za9iuttZc2Hn40yZ0dagMAAGAGTOIaz7+X5Jcm8HMAAACYQWMFz6r6kSQvJXnfTfY5VlWrVbV68eLFcQ4HAADAAO04eFbV25O8Ocn3tNbajfZrrZ1qra201laWlpZ2ejgAAAAGav9Ovqmq7k/yriTf0Vr7w8mWBAAAwCwZ5XYq70/ykSSHqurZqnpnkp9M8ookj1XVU1X1ns51AgAAMFDbzni21t62xeaf6VALAAAAM2gSXW0BAADghgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKArwRMAAICuBE8AAAC6EjwBAADoSvAEAACgK8ETAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK4ETwAAALoSPAEAAOhK8AQAAKCrbYNnVT1SVc9X1aeu2fbHq+qxqvqdjf9f2bdMAAAAhmqUGc9Hk9y/aduDSX61tfbqJL+68RgAAAC+yrbBs7X2RJIvbNr81iQ/u/H1zyY5OuG6AAAAmBE7vcbzG1prn0+Sjf+//kY7VtWxqlqtqtWLFy/u8HAAAAAMVffmQq21U621ldbaytLSUu/DAQAAsMfsNHj+z6r6piTZ+P/5yZUEAADALNlp8PxQkrdvfP32JL8wmXIAAACYNaPcTuX9ST6S5FBVPVtV70zycJI3VNXvJHnDxmMAAAD4Kvu326G19rYbPPX6CdcCAADADOreXAgAAID5JngCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF3tH+ebq+qfJvn7SVqS80ne0Vr7f5MoDAC43plzazl59kKeu7SeA4sLOX7kUI4eXt7tsgBgWzue8ayq5ST/JMlKa+1bk+xL8t2TKgwAeNmZc2s5cfp81i6tpyVZu7SeE6fP58y5td0uDQC2Ne5S2/1JFqpqf5Lbkzw3fkkAwGYnz17I+ouXr9u2/uLlnDx7YZcqAoDR7Th4ttbWkvzLJM8k+XySP2it/crm/arqWFWtVtXqxYsXd14pAMyx5y6t39J2ANhLxllq+8okb01yd5IDSb62qr53836ttVOttZXW2srS0tLOKwWAOXZgceGWtgPAXjLOUtvvTPK51trF1tqLSU4n+SuTKQsAuNbxI4eycNu+67Yt3LYvx48c2qWKAGB043S1fSbJt1fV7UnWk7w+yepEqgIArnO1e62utgAM0Y6DZ2vtY1X1wSQfT/JSknNJTk2qMADgekcPLwuaAAzSWPfxbK29O8m7J1QLAAAAM2jc26kAAADATQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXYwXPqlqsqg9W1W9V1dNV9ZcnVRgAAACzYf+Y3/8TSX65tfZdVfU1SW6fQE0AAADMkB0Hz6r6o0n+epK/mySttS8l+dJkygIAAGBWjLPU9k8luZjk31fVuap6b1V97eadqupYVa1W1erFixfHOBwAAABDNE7w3J/k25L8dGvtcJIvJnlw806ttVOttZXW2srS0tIYhwMAAGCIxgmezyZ5trX2sY3HH8yVIAoAAABfsePg2Vr7H0l+v6oObWx6fZJPT6QqAAAAZsa4XW1/IMn7NjrafjbJO8YvCQAAgFkyVvBsrT2VZGVCtQAAADCDxrnGEwAAALYleAIAANCV4AkAAEBXgicAAABdCZ4AAAB0JXgCAADQleAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF0JngAAAHQleAIAANCV4AkAAEBXgicAAABd7d/tAgAAgJs7c24tJ89eyHOX1nNgcSHHjxzK0cPLu10WjEzwBACAPezMubWcOH0+6y9eTpKsXVrPidPnk0T4ZDAstQUAgD3s5NkLXwmdV62/eDknz17YpYrg1gmeAACwhz13af2WtsNeNPZS26ral2Q1yVpr7c3jlwQAAFx1YHEha1uEzAOLC7tQzexw3ex0TWLG8weTPD2BnwMAAGxy/MihLNy277ptC7fty/Ejh3apouG7et3s2qX1tLx83eyZc2u7XdrMGit4VtWdSf5mkvdOphwAAOBaRw8v56EH7sny4kIqyfLiQh564B6zc2Nw3ez0jbvU9seT/HCSV9xoh6o6luRYktx1111jHg4AAObP0cPLguYEuW52+nY841lVb07yfGvtyZvt11o71Vpbaa2tLC0t7fRwAAAAE3Gj62NdN9vPOEtt703ylqr63SQfSHJfVf2HiVQFAADQietmp2/HS21bayeSnEiSqnpdkh9qrX3vhOoCAAAGaAjdYq/Ws9frvGoIr+l2xr6dCgAAQPJyt9irjXuudotNsueC0lCumx3Sa3ozEwmerbUPJ/nwJH4WADBss3BmnvEYA/PrZt1ijYGdmZXX1IwnADAxs3Jmnp0zBuabbrGTNyuv6Vj38QQAuNbQ7o135txa7n348dz94C/m3ocfd/P4CRjaGGCydIudvFl5TQVPAGBihnRm/urM3Nql9bS8PDMnfI5nSGOAydMtdvJm5TUVPAGAiRnSmXkzc30MaQwweUcPL+ehB+7J8uJCKsny4kIeeuAey6zHMCuvqWs8AYCJOX7k0HXX9yV798z80GbmhtKwZ0hjgD6G0i12SGbhNRU8AYCJGdK98Q4sLmRti5C5F2fmhtSwZ0hjAJieaq1N7WArKyttdXV1ascDALiRzWEuuTIztxeXsN378ONbhuTlxYX8+oP37UJFAFurqidbayubt5vxBADm0pBm5oa2LBhgM8ETAJhbQ7luakjLggG2oqstAMAeNyu3UwDmlxlPAIA9bkjLggG2InhChtOiHpg873+GYijLggG2Ingy94bUot4HZJisIb3/AWDIXOPJ3Dt59sJ1rfSTZP3Fyzl59sIuVbS1qx+Q1y6tp+XlD8hnzq3tdmkwWEN5/wPA0JnxZO4NpUX9zT4gm5kZj5nk+TWU9z8ADJ0ZT+bejVrR77UW9T4g92Emeb4N5f0PAEMneDL3htKi3gfkPiy1nG9Def8DwNBZasvcG0qL+uNHDl3XBCXxAXkSzCT3MZTly0N5/wPA0AmekGG0qPcBuY8DiwtZ2yJkmkneuaF1ih3C+x8Ahk7whAHxAXnyhjaTPISZRI2wAIDNdhw8q+pVSX4uyTcm+XKSU621n5hUYQDTMKSZ5KHMJFq+DABsNs6M50tJ/llr7eNV9YokT1bVY621T0+oNoCpGMpM8lBmEi1fBgA223FX29ba51trH9/4+v8meTrJ3vnkc4vOnFvLvQ8/nrsf/MXc+/DjbqUA7DlDmUnUKRYA2Gwi13hW1cEkh5N8bIvnjiU5liR33XXXJA43cUNZvjY0Q7gWDYZkKDOJQ1q+DABMR7XWxvsBVV+X5NeS/PPW2umb7buystJWV1fHOl4P9z78+JYf5pYXF/LrD963CxUN3+Ywn1yZ8XjogXt8+IQd8r4CAPa6qnqytbayefuOl9pu/NDbkvx8kvdtFzr3sqEsXxuSm12LBuzM0cPLeeiBe7K8uJDKlZNjQicAMATjdLWtJD+T5OnW2r+eXEnTN5Tla0MizEMfQ2mEBABwrXFmPO9N8n1J7quqpzb+vWlCdU2VRhiTd6PQLswDAMD82fGMZ2vtvyapCdayazTCmLzjRw5teS2aMA8AAPNnIl1tZ8FQlq8NpVOsMA8AAFwleA7I0G77MpQwTx9DOUkCAEB/Y3W1Zbp0imUorp4kWbu0npaXT5KcObe226UBALALBM8B0SmWoXCSBACAawmeA6JTLEPhJAkAANcSPAfEbV8YCidJAAC4luA5IEcPL+ehB+7J8uJCKsny4kIeeuAeDVvYc5wkAQDgWrraDoxOsQyB2+kAAHAtwRPowkkSAACustQWAACArgRPAAAAuhI8AQAA6ErwBAAAoCvBEwAAgK6qtTa9g1VdTPJ7UzvgztyR5IXdLoK5ZxyyFxiH7BXGInuBccheMIRx+Cdba0ubN041eA5BVa221lZ2uw7mm3HIXmAcslcYi+wFxiF7wZDHoaW2AAAAdCV4AgAA0JXg+dVO7XYBEOOQvcE4ZK8wFtkLjEP2gsGOQ9d4AgAA0JUZTwAAALoSPAEAAOhqLoNnVd1fVReq6jNV9eAWz1dV/duN5z9ZVd+2G3Uy20YYh9+zMf4+WVW/UVWv3Y06mX3bjcVr9vsLVXW5qr5rmvUxH0YZh1X1uqp6qqp+s6p+bdo1MvtG+Nv8x6rqP1XVJzbG4Tt2o05mW1U9UlXPV9WnbvD8ILPK3AXPqtqX5N8leWOSb0nytqr6lk27vTHJqzf+HUvy01Mtkpk34jj8XJLvaK29JsmPZcAXk7N3jTgWr+73L5KcnW6FzINRxmFVLSb5qSRvaa39uSR/e+qFMtNG/H34j5J8urX22iSvS/Kvquprploo8+DRJPff5PlBZpW5C55J/mKSz7TWPtta+1KSDyR566Z93prk59oVH02yWFXfNO1CmWnbjsPW2m+01v73xsOPJrlzyjUyH0b5nZgkP5Dk55M8P83imBujjMO/k+R0a+2ZJGmtGYtM2ijjsCV5RVVVkq9L8oUkL023TGZda+2JXBlbNzLIrDKPwXM5ye9f8/jZjW23ug+M41bH2DuT/FLXiphX247FqlpO8reSvGeKdTFfRvmd+KeTvLKqPlxVT1bV90+tOubFKOPwJ5P82STPJTmf5Adba1+eTnnwFYPMKvt3u4BdUFts23xPmVH2gXGMPMaq6m/kSvD8q10rYl6NMhZ/PMm7WmuXr5zkh4kbZRzuT/Lnk7w+yUKSj1TVR1trv927OObGKOPwSJKnktyX5JuTPFZV/6W19n96FwfXGGRWmcfg+WySV13z+M5cOWt1q/vAOEYaY1X1miTvTfLG1tr/mlJtzJdRxuJKkg9shM47krypql5qrZ2ZTonMgVH/Nr/QWvtiki9W1RNJXptE8GRSRhmH70jycGutJflMVX0uyZ9J8t+mUyIkGWhWmceltv89yaur6u6Ni8G/O8mHNu3zoSTfv9Ex6tuT/EFr7fPTLpSZtu04rKq7kpxO8n3O6NPRtmOxtXZ3a+1ga+1gkg8m+YdCJxM2yt/mX0jy16pqf1XdnuQvJXl6ynUy20YZh8/kyqx7quobkhxK8tmpVgkDzSpzN+PZWnupqv5xrnRm3Jfkkdbab1bVP9h4/j1J/nOSNyX5TJI/zJWzWzAxI47DH03yJ5L81MZM00uttZXdqpnZNOJYhK5GGYettaer6peTfDLJl5O8t7W25a0GYCdG/H34Y0kerarzubLc8V2ttRd2rWhmUlW9P1e6Jt9RVc8meXeS25JhZ5W6slIAAAAA+pjHpbYAAABMkeAJAABAV4InAAAAXQmeAAAAdCV4AgAA0JXgCQAAQFeCJwAAAF39fy1OUk9E2khEAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "np.random.seed(0)\n", "X = np.linspace(0, 1, 20)\n", "y = 3*X + 1 + 0.5*np.random.randn(20)\n", "X = X.reshape(-1, 1)\n", "\n", "y[10] = 8\n", "y[15] = 15\n", "\n", "plt.figure(figsize=(16, 4))\n", "plt.scatter(X, y)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A simple linear regression will not do a good job since it is distracted by the outliers. That is because it optimizes the mean squared error\n", "$$ \\sum_i \\left(y_i-\\textrm{model}(x_i)\\right)^2 $$\n", "\n", "which penalizes a few large errors more than many tiny errors. For example, if y-model(x) = 4 for some single observation, the MSE here is 16. If there are two observations with y_1 - model(x_1) = 2 and y_2 - model(x_2) = 2, the MSE is 8 in total, which is less than for one larger error. Note that the sum of the errors is the same in both cases.\n", "\n", "Hence, linear regression does the following:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[]" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA54AAAD4CAYAAACEyjk9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAbWElEQVR4nO3de5Cd50Hf8d+j1cWr6+qWi2XLMo4QjqM4Sraum7QlJWUcKCUupTOk3Jqm4+m0hUxbDHGZgT+YDumoF+hQYFxIA1Mm/BFcQ2cAkYGBtMGhlXGIA0aQhhIsp1i2dbGt1W319I9dIR15z+45u+c9u+85n8+MRrvvHj3nOTuPpP2e9z3PKbXWAAAAQFPWrfYEAAAAGG3CEwAAgEYJTwAAABolPAEAAGiU8AQAAKBR64d5Z3v27KkHDhwY5l0CAAAwJE8++eQLtda9Nx8fangeOHAgx48fH+ZdAgAAMCSllD9d6LhLbQEAAGiU8AQAAKBRwhMAAIBGCU8AAAAaJTwBAABo1FB3tQUAgLXi8adO5uixE3nuzExunZrMww8cyoNH9q32tGAkCU8AAMbO40+dzCOPPZ2Zy7NJkpNnZvLIY08nifiEBrjUFgCAsXP02Im/iM5rZi7P5uixE6s0IxhtwhMAgLHz3JmZvo4DKyM8AQAYO7dOTfZ1HFgZ4QkAwNh5+IFDmdww0XFscsNEHn7g0CrNCEabzYUAABg71zYQsqstDIfwBABgLD14ZJ/QhCFxqS0AAACNEp4AAAA0SngCAADQKOEJAABAo4QnAAAAjRKeAAAANEp4AgAA0Kglw7OU8tFSyvOllM8v8LXvKaXUUsqeZqYHAABA2/VyxvNjSd5788FSyu1JvjbJlwY8JwAAAEbIkuFZa/1UkpcW+NJ/SPK9SeqgJwUAAMDoWNZrPEsp35jkZK3193q47UOllOOllOOnTp1azt0BAADQYn2HZyllc5LvT/IDvdy+1vporXW61jq9d+/efu8OAACAllvOGc+7ktyZ5PdKKf83yW1JfreU8oZBTgwAAIDRsL7fP1BrfTrJ6659Ph+f07XWFwY4LwAAAEZEL2+n8vEkTyQ5VEp5tpTyweanBQAAwKhY8oxnrfX9S3z9wMBmAwAAwMhZ1q62AAAA0CvhCQAAQKOEJwAAAI0SngAAADRKeAIAANAo4QkAAECjhCcAAACNEp4AAAA0SngCAADQKOEJAABAo4QnAAAAjRKeAAAANEp4AgAA0CjhCQAAQKOEJwAAAI0SngAAADRKeAIAANAo4QkAAECjhCcAAACNWjI8SykfLaU8X0r5/A3HjpZS/rCU8rlSyn8rpUw1O00AAADaqpcznh9L8t6bjn0yyVtqrW9N8kdJHhnwvAAAABgRS4ZnrfVTSV666div1VqvzH/6mSS3NTA3AAAARsAgXuP5D5P8ygDGAQAAYAStKDxLKd+f5EqSn1vkNg+VUo6XUo6fOnVqJXcHAABACy07PEsp35nkG5J8a621drtdrfXRWut0rXV67969y707AAAAWmr9cv5QKeW9Sb4vyVfXWs8PdkoAAACMkl7eTuXjSZ5IcqiU8mwp5YNJfizJtiSfLKV8tpTykw3PEwAAgJZa8oxnrfX9Cxz+6QbmAgAAwAgaxK62AAAA0JXwBAAAoFHCEwAAgEYJTwAAABolPAEAAGiU8AQAAKBRwhMAAIBGCU8AAAAaJTwBAABolPAEAACgUcITAACARglPAAAAGiU8AQAAaJTwBAAAoFHCEwAAgEYJTwAAABolPAEAAGiU8AQAAKBRwhMAAIBGCU8AAAAatWR4llI+Wkp5vpTy+RuO7SqlfLKU8sfzv+9sdpoAAAC0VS9nPD+W5L03Hftwkl+vtR5M8uvznwMAAMBrLBmetdZPJXnppsPvS/Iz8x//TJIHBzwvAAAARsRyX+P5+lrrl5Nk/vfXdbthKeWhUsrxUsrxU6dOLfPuAAAAaKvGNxeqtT5aa52utU7v3bu36bsDAABgjVlueP55KeWNSTL/+/ODmxIAAACjZLnh+UtJvnP+4+9M8ouDmQ4AAACjppe3U/l4kieSHCqlPFtK+WCSjyT52lLKHyf52vnPAQAA4DXWL3WDWuv7u3zpPQOeCwAAACOo8c2FAAAAGG/CEwAAgEYJTwAAABolPAEAAGiU8AQAAKBRwhMAAIBGCU8AAAAaJTwBAABolPAEAACgUcITAACARglPAAAAGiU8AQAAaJTwBAAAoFHCEwAAgEYJTwAAABolPAEAAGiU8AQAAKBRwhMAAIBGCU8AAAAaJTwBAABo1PqV/OFSyj9P8o+S1CRPJ/lArfXCICYGAHR6/KmTOXrsRJ47M5Nbpybz8AOH8uCRfas9LQBY0rLPeJZS9iX57iTTtda3JJlI8i2DmhgAcN3jT53MI489nZNnZlKTnDwzk0ceezqPP3VytacGAEta6aW265NMllLWJ9mc5LmVTwkAuNnRYycyc3m249jM5dkcPXZilWYEAL1bdnjWWk8m+bdJvpTky0nO1lp/7ebblVIeKqUcL6UcP3Xq1PJnCgBj7LkzM30dB4C1ZCWX2u5M8r4kdya5NcmWUsq33Xy7WuujtdbpWuv03r17lz9TABhjt05N9nUcANaSlVxq+zeT/Emt9VSt9XKSx5K8czDTAgBu9PADhzK5YaLj2OSGiTz8wKFVmhEA9G4lu9p+Kcn9pZTNSWaSvCfJ8YHMCgDocG33WrvaAtBGyw7PWuvvlFI+keR3k1xJ8lSSRwc1MQCg04NH9glNAFppRe/jWWv9wSQ/OKC5AAAAMIJW+nYqAAAAsCjhCQAAQKOEJwAAAI0SngAAADRKeAIAANAo4QkAAECjhCcAAACNEp4AAAA0SngCAADQKOEJAABAo4QnAAAAjRKeAAAANEp4AgAA0CjhCQAAQKOEJwAAAI0SngAAADRKeAIAANAo4QkAAECjhCcAAACNEp4AAAA0akXhWUqZKqV8opTyh6WUZ0opf2VQEwMAAGA0rF/hn//RJL9aa/3mUsrGJJsHMCcAAABGyLLDs5SyPclfT/IPkqTWeinJpcFMCwAAgFGxkkttvyLJqST/pZTyVCnlp0opW26+USnloVLK8VLK8VOnTq3g7gAAAGijlYTn+iRvT/ITtdYjSV5N8uGbb1RrfbTWOl1rnd67d+8K7g4AAIA2WslrPJ9N8myt9XfmP/9EFghPAAAA+nD+fPLCC8mLL3b+PjubfOhDqz27ZVl2eNZa/18p5c9KKYdqrSeSvCfJHwxuagAAAC1Wa/eI7Pbxiy8mMzMLjzc1NX7hOe+7kvzc/I62X0zygZVPCQAAYI2pNXn11f4j8sKF7mPu2pXs3p3s2ZPcfnvytrfNfXzt2I0f7949d/uWWlF41lo/m2R6QHMBAABoXq3JK6/0H5EXLy48XimdEXnHHcnb3754RO7cmaxf6XnA9hifRwoAAIyeWpOXX+4/Ii91eSfIUubC8Fok3nlnMj3dPSL37Jm7BHZiYriPu2WEJwAAsDbUmpw7t3gwLnTs8uWFx1u3rjMi77orue++pSNy3Ure/IOFCE8AAGDwak3Onu0/Iq9cWXi8iYnOiDx4MLn//oUvY712bMcOEblGCE8AAGBxV6/2H5EvvbR4RN4YjIcOJe985+IRuX27iGwx4QkAAOPk6tXkzJn+I3J2duHx1q/vDMa77+6+oc6NEVnKcB83q0p4AgBAW129mpw+3X9EXr268HgbNnQG4z33LB2R27aJSJYkPAEAYC2YnV1eRNa68HgbN3YG4+HDS0fk1q0ikkYITwAAGLTZ2bko7CciT5/uHpGbNnUG4733dt+V9drHW7aISNYM4QkAAIu5cmV5EdnNLbd0RuL+/UtH5ObNIpJWE54AAIyPy5f7j8gzZ7qPNznZGYkHDnS/jPXGiIQxIzwBAGiny5cXD8aFgvLs2e7jbd7cGYlf8RWLR+Tu3SISeiQ8AQBYfZcu9R+R5851H2/Lls5IfNOblo7IycnhPV4YM8ITAIDBunix/4h8+eXu423b1hmMX/mVS0fkLbcM7/ECSxKeAAB0d+FC/xH5yivdx9u+vTMUv+qrlo7ITZuG93iBRghPAIBxMTPT+4Y6135/9dXu4+3YcT0UX/e65M1v7r6hzrWI3LhxeI8XWDOEJwBAG50/339Enj/ffbypqeuR+IY3JG95y+IRuWuXiAR6JjwBAFZTrcuLyJmZ7mPu3Hk9Em+9NXnrW5eOyA0bhveY6dvjT53M0WMn8tyZmdw6NZmHHziUB4/sW+1pQc+EJwDAoNQ6d2lqvxF54UL3MXftuh6Jt9+evO1tS0fkej/ijZLHnzqZRx57OjOXZ5MkJ8/M5JHHnk4S8Ulr+FcJAGAhtc5tktNvRF68uPB4pXRG5B13JG9/++IRuXOniCRHj534i+i8ZubybI4eOyE8aQ3/kgEAo6/Wubfr6DciL11aeLxSrm+Ws2dPcuedyfT00hE5MTHcx81IeO7MwpdVdzsOa9GKw7OUMpHkeJKTtdZvWPmUAAAWUWty7lz/EXn58sLjrVvXGZF33ZXcd1/3iNyzZ24jnnXrhvu4GVu3Tk3m5AKReevU5CrMZnR43exwDeKM54eSPJNk+wDGAgDGSa3J2bP9R+SVKwuPNzHRGZEHDyb33794RO7YISJZ0x5+4FDHazyTZHLDRB5+4NAqzqrdvG52+FYUnqWU25L8rST/Osm/GMiMAIB2unq1/4h88cXFI/LGSDx0qPtlrNeObd8uIhk510LI2bnB8brZ4VvpGc8fSfK9SbZ1u0Ep5aEkDyXJ/v37V3h3AMBQXL2anDnTe0S+8ELy0kvJ7OzC461f3xmMd9/dW0SWMtzHDWvUg0f2CaIB8rrZ4Vt2eJZSviHJ87XWJ0sp7+52u1rro0keTZLp6em63PsDAJbp6tXk9On+I/Lq1YXH27ChMxjvuWfpiNy2TUQCa4bXzQ7fSs54vivJN5ZSvj7JLUm2l1L+a6312wYzNQDgNWZn+4/I06e7R+TGjZ3BePjw0hG5dauIBFrN62aHb9nhWWt9JMkjSTJ/xvN7RCcA9GF2du7MYr8RWbtcQLRpU2cw3ntv9w11rn28ZYuIBAaqDbvFtu11s234ni7F+3gCwCBcudI9IrsF5Zkz3SPylls6I/HIkaUjcvNmEQmsqjbtFtuW18226Xu6mIGEZ631N5P85iDGAoBVd/ny8iKym8nJzki8447eIrKlRuGZeVbGGhhfdosdvFH5njrjCcBou3z5+tt29BqRZ892H2/z5s5IvPPOxSNy9+5WR2S/RuWZeZbPGhhvdosdvFH5ngpPANrj0qXXhuJSEXnuXPfxtmzpjMS77uq+oc61iJy04+Fi2vbMvDNzg9e2NcBg2S128Ebleyo8AVgdFy/2H5Evv9x9vK1bOyPx4MGlI/KWW4b3eMdEm56Zd2auGW1aAwye3WIHb1S+p8ITgJW7cKH/iHzlle7jbd/eGYyHDi0dkZs2De/x0lWbnpl3Zq4ZbVoDDF7bdottg1H5ngpPADpduNDbW3vceOzVV7uPt2PH9VDcuze5++6lI3LjxuE9XgaqTc/Mt+3MXFsuC27TGqAZbdkttk1G4XsqPAFG2cxM/xF5/nz38aamrofi61+f3HPP4hG5a5eIHDNtema+TWfm2nRZcJvWADA8pXZ7/7AGTE9P1+PHjw/t/gBGyvnzvUXkjR/PLHLmZufO7sF488fXInLDhuE9XmjYzTGXzJ2Z++FvOrzmIuldH/mNBSN539RkPv3hr1mFGQEsrJTyZK11+ubjzngCDFutvUfkjccuXOg+5q5d1yPxttuSt71t6Yhc778Axlubzsy17bJggJv5qQNgJWqde31jvxF58eLC45XSGZH79ydHjiwekTt3ikhYpra8bqpNlwUDLMRPKgDX1Dq302q/l7NeurTweKVc3yxnz57kwIHkHe9YOiInJob6sIG1z4Y9QNsJT2A01Tr3no/9RuTlywuPt25dZ0TedVdy332LR+TUlIgEBqJNlwUDLER4QtqzRf3YqjU5d66/iHzxxe4ROTHRGZEHDyb337/4Jjs7dszFJyPH33/aoi2XBQMsRHgy9tq0Rf1I/IBca3L2bP8ReeXKwuNNTHRG4qFDS+/Uun27iCRJu/7+A0CbCU/G3tFjJzpeM5MkM5dnc/TYiTX1g+ea/AH56tWFI3KxoHzxxWR2duHx1q/vjMS77+5+GeuNEVnKcB83I6Mtf/8BoO2EJ2OvLVvUN/4D8tWryZkz/Ufk1asLj7dhQ2ck3nPP0hG5bduqRORInElmWdry9x8A2k54MvbaskV9Xz8gz872H5EvvdQ9Ijdu7AzGw4eXjsitW1txJnJNnklmaNry9x8A2k54MvbW/Bb1s7PJ6dO5/9KpXH7++eyceTk7z5/LrplzmZo5l9tmzyfve/S1EVnrwuNt2tQZjPfeu3REbtnSiohcDpdajrc1//cfAEaE8GTsDXWL+itX5qJwsQ11bj52+nRSaz6+wHAX1m/M1d27k9nXz4XikSPdN9S59vHmzSMbkcvhUstmtOXyZW9RAQDDITwhy9yi/sqV669z7Cciu5mc7IzEO+54TTh++kzyn585lz++sjGb3vD6fPffvtcPyCvkUsvBa9vly96iAgCaJzwhmXu/x34j8syZ7uNt3twZkXfeufiZyN275/7MEt41/4vBadullm04k+jyZQDgZssOz1LK7Ul+NskbklxN8mit9UcHNTFYtkuX+o/Is2e7j7dlS2ck3nXX0hE56WxZW7TpUsu2nEl0+TIAcLOVnPG8kuRf1lp/t5SyLcmTpZRP1lr/YEBzg+Tixf4j8ty57uNt3doZiQcPdt9Q51pE3nLL8B4vq6Itl1q25Uyiy5cBgJstOzxrrV9O8uX5j18upTyTZF+SVoZnGy5fa72LFxcPxoWC8uWXu4+3bVtnJB46tHREbto0vMcLA9aWM4ltu3wZAGjeQF7jWUo5kORIkt9Z4GsPJXkoSfbv3z+Iuxu4tly+tqZcuLBkRP75F0/m9J99OVtfPpNdF17O5kuL/HC8Y8f1UNy7N7n77qUjcuPG4T1eWAPaciaxTZcvAwDDUWq39/rrdYBStib5rST/utb62GK3nZ6ersePH1/R/TXhXR/5jQV/mNs3NZlPf/hrVmFGQzYz0/+ZyFdf7T7e1FRe2boj/6fekhc3bcvpzdvz0uT2vLJlR979rrtz5MibOiNy1y4RCT24+UmyZO5M4g9/02FRBwCsCaWUJ2ut0zcfX9EZz1LKhiS/kOTnlorOtawtl6/15Pz5/iPy/Pnu4+3cef1s4xvfmBw+vPiZyF27kg0b8kCXmP/E1sl8+u+OQcxDA5xJBADaaiW72pYkP53kmVrrvx/clIZvTV6+VuvyInJmkVjetet6KO7bl9x778LxeGNErl/eEhmpmIc1pC0bIQEA3GglZzzfleTbkzxdSvns/LF/VWv95ZVPa7ga3wij1rlLU/uNyAsXFh6vlM6I3L8/OXJk8YjcuXPZEbkcazLmAQCAVbGSXW3/Z5IywLmsmr4uX6s1eeWV/iPy4sWF77yU65vl7NmTHDiQvOMdS0fkxERz35ABsKslAABwzYo3F+rHWt1cKEly5kzyxBO9ReSlSwuPsW5dZ0Qu9lrIa79PTfUVkW1625c2zRUAAFi5bpsLCc9rnngieec7r38+MbG8iFy3rrEp2tGSNvHEAwDA+GlkV9uRcvjwXHxeC8rt2xuNyOU4euxER3Qmyczl2Rw9dsIP9Kwp3hsXAIAbra2yWk1btyb335+86U2Nn7lcLjvF0haLPUkCAMD4WXt1RVfddoS1UyxrjSdJAAC4kfBskYcfOJTJDZ0bEdkplrXIkyQAANxIeLbIg0f25Ye/6XD2TU2mJNk3NWljIdYkT5IAAHAjmwu1zINH9glN1ry+3hsXAICRJzyBRniSBACAa1xqCwAAQKOEJwAAAI0SngAAADRKeAIAANAo4QkAAECjSq11eHdWyqkkfzq0O1yePUleWO1JMPasQ9YC65C1wlpkLbAOWQvasA7vqLXuvfngUMOzDUopx2ut06s9D8abdchaYB2yVliLrAXWIWtBm9ehS20BAABolPAEAACgUcLztR5d7QlArEPWBuuQtcJaZC2wDlkLWrsOvcYTAACARjnjCQAAQKOEJwAAAI0ay/Aspby3lHKilPKFUsqHF/h6KaX8x/mvf66U8vbVmCejrYd1+K3z6+9zpZTfLqXcuxrzZPQttRZvuN1fKqXMllK+eZjzYzz0sg5LKe8upXy2lPL7pZTfGvYcGX09/N+8o5Ty30spvze/Dj+wGvNktJVSPlpKeb6U8vkuX29lq4xdeJZSJpL8pyRfl+TNSd5fSnnzTTf7uiQH5389lOQnhjpJRl6P6/BPknx1rfWtSX4oLX4xOWtXj2vx2u3+TZJjw50h46CXdVhKmUry40m+sdZ6T5K/N/SJMtJ6/Pfwnyb5g1rrvUneneTflVI2DnWijIOPJXnvIl9vZauMXXgmuS/JF2qtX6y1Xkry80ned9Nt3pfkZ+uczySZKqW8cdgTZaQtuQ5rrb9daz09/+lnktw25DkyHnr5NzFJvivJLyR5fpiTY2z0sg7/fpLHaq1fSpJaq7XIoPWyDmuSbaWUkmRrkpeSXBnuNBl1tdZPZW5tddPKVhnH8NyX5M9u+PzZ+WP93gZWot819sEkv9LojBhXS67FUsq+JH8nyU8OcV6Ml17+TfzKJDtLKb9ZSnmylPIdQ5sd46KXdfhjSe5O8lySp5N8qNZ6dTjTg7/QylZZv9oTWAVlgWM3v6dML7eBleh5jZVS/kbmwvOvNjojxlUva/FHknxfrXV27kl+GLhe1uH6JO9I8p4kk0meKKV8ptb6R01PjrHRyzp8IMlnk3xNkruSfLKU8j9qreeanhzcoJWtMo7h+WyS22/4/LbMPWvV721gJXpaY6WUtyb5qSRfV2t9cUhzY7z0shank/z8fHTuSfL1pZQrtdbHhzNFxkCv/ze/UGt9NcmrpZRPJbk3ifBkUHpZhx9I8pFaa03yhVLKnyT5qiT/azhThCQtbZVxvNT2fyc5WEq5c/7F4N+S5Jduus0vJfmO+R2j7k9yttb65WFPlJG25DospexP8liSb/eMPg1aci3WWu+stR6otR5I8okk/0R0MmC9/N/8i0n+WillfSllc5K/nOSZIc+T0dbLOvxS5s66p5Ty+iSHknxxqLOElrbK2J3xrLVeKaX8s8ztzDiR5KO11t8vpfzj+a//ZJJfTvL1Sb6Q5Hzmnt2CgelxHf5Akt1Jfnz+TNOVWuv0as2Z0dTjWoRG9bIOa63PlFJ+NcnnklxN8lO11gXfagCWo8d/D38oycdKKU9n7nLH76u1vrBqk2YklVI+nrldk/eUUp5N8oNJNiTtbpUyd6UAAAAANGMcL7UFAABgiIQnAAAAjRKeAAAANEp4AgAA0CjhCQAAQKOEJwAAAI0SngAAADTq/wMQd8/bwSZXkAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "from sklearn.linear_model import LinearRegression\n", "\n", "x = np.array([0, 1]).reshape(-1, 1)\n", "plt.figure(figsize=(16, 4))\n", "plt.scatter(X, y)\n", "plt.plot(x, LinearRegression().fit(X, y).predict(x), 'r')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "By changing the loss function to the mean absolute deviation \n", "$$ \\sum_i \\left|y_i-\\textrm{model}(x_i)\\right|\\enspace, $$\n", "\n", "we can let the model put the same focus on each error. This yields the least absolute deviation (LAD) regression that tries to agree with the majority of the points." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[]" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA54AAAD4CAYAAACEyjk9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3df3Dkd33f8ddb+0Na7erH6YfvfLqfPhthjH0+o7ouTgqFMmcowS6lM6H5QV06TqctYZrigpOZkBmageT6i05CqEscwpRx2hDXIR3gQmEIDcGkZ2w4gjlwDTa+c3y6H7rTSitpf7z7x0p3Wt3+3v2u9rv7fMzc3Oqrve9+pPla1nO/n+/na+4uAAAAAACCMrDdAwAAAAAA9DbCEwAAAAAQKMITAAAAABAowhMAAAAAECjCEwAAAAAQqGgnX2xqasoPHDjQyZcEAAAAAHTIk08+ec7dp7du72h4HjhwQCdOnOjkSwIAAAAAOsTMni+3nam2AAAAAIBAEZ4AAAAAgEARngAAAACAQBGeAAAAAIBAEZ4AAAAAgEB1dFVbAAAAoFs8/tRpHTt+SmcWMto9ntCDR2d135GZ7R4W0JMITwAAAPSdx586rYceO6lMNi9JOr2Q0UOPnZQk4hMIAFNtAQAA0HeOHT91JTo3ZLJ5HTt+aptGBPQ2whMAAAB958xCpqHtAFpDeAIAAKDv7B5PNLQdQGsITwAAAPSdB4/OKhGLlGxLxCJ68OjsNo0I6G0sLgQAAIC+s7GAEKvaAp1BeAIAAKAv3XdkhtAEOoSptgAAAACAQBGeAAAAAIBAEZ4AAAAAgEARngAAAACAQBGeAAAAAIBAEZ4AAAAAgEARngAAAACAQNUMTzN7xMzOmtl3ynzufWbmZjYVzPAAAAAAAGFXzxnPT0q6Z+tGM9sr6U2SXmjzmAAAAAAAPaRmeLr7VyVdKPOp/yjp30jydg8KAAAAANA7mrrG08zeJum0u3+rjuc+YGYnzOzE/Px8My8HAAAAAAixhsPTzIYl/YqkX63n+e7+sLvPufvc9PR0oy8HAAAAAAi5Zs54HpJ0UNK3zOxHkvZI+qaZ7WrnwAAAAAAAvSHa6D9w95OSrtv4eD0+59z9XBvHBQAAAADoEfXcTuVRSV+XNGtmL5rZu4MfFgAAAACgV9Q84+nu76zx+QNtGw0AAAAAoOc0taotAAAAAAD1IjwBAAAAAIEiPAEAAAAAgSI8AQAAAACBIjwBAAAAAIEiPAEAAAAAgSI8AQAAAACBIjwBAAAAAIEiPAEAAAAAgSI8AQAAAACBIjwBAAAAAIEiPAEAAAAAgSI8AQAAAACBIjwBAAAAAIEiPAEAAAAAgSI8AQAAAACBIjwBAAAAAIEiPAEAAAAAgSI8AQAAAACBqhmeZvaImZ01s+9s2nbMzL5nZt82s/9pZuPBDhMAAAAAEFb1nPH8pKR7tmz7oqRXu/ttkr4v6aE2jwsAAAAA0CNqhqe7f1XShS3b/tTdc+sfPiFpTwBjAwAAAAD0gHZc4/lPJH2+DfsBAAAAAPSglsLTzH5FUk7Sp6s85wEzO2FmJ+bn51t5OQAAAABACDUdnmb2LklvlfQz7u6VnufuD7v7nLvPTU9PN/tyAAAAAICQijbzj8zsHknvl/Q6d19u75AAAAAAAL2kntupPCrp65JmzexFM3u3pN+SNCLpi2b2tJl9POBxAgAAAABCquYZT3d/Z5nNvxvAWAAAAAAAPagdq9oCAAAAAFAR4QkAAAAACBThCQAAAAAIFOEJAAAAAAgU4QkAAAAACBThCQAAAAAIFOEJAAAAAAgU4QkAAAAACBThCQAAAAAIFOEJAAAAAAgU4QkAAAAACBThCQAAAAAIFOEJAAAAAAgU4QkAAAAACBThCQAAAAAIFOEJAAAAAAgU4QkAAAAACBThCQAAAAAIFOEJAAAAAAgU4QkAAAAACFTN8DSzR8zsrJl9Z9O2CTP7opn9YP3vHcEOEwAAAAAQVvWc8fykpHu2bPuApC+5+02SvrT+MQAAAAAA16gZnu7+VUkXtmy+V9Lvrz/+fUn3tXlcAAAAAIAe0ew1njvd/SVJWv/7ukpPNLMHzOyEmZ2Yn59v8uUAAAAAAGEV+OJC7v6wu8+5+9z09HTQLwcAAAAA6DLNhufLZna9JK3/fbZ9QwIAAAAA9JJmw/Ozkt61/vhdkv64PcMBAAAAAPSaem6n8qikr0uaNbMXzezdkj4i6U1m9gNJb1r/GAAAAACAa0RrPcHd31nhU29s81gAAAAAAD0o8MWFAAAAAAD9jfAEAAAAAASK8AQAAAAABIrwBAAAAAAEivAEAAAAAASK8AQAAAAABIrwBAAAAAAEivAEAAAAAASK8AQAAAAABIrwBAAAAAAEivAEAAAAAASK8AQAAAAABIrwBAAAAAAEivAEAAAAAASK8AQAAAAABIrwBAAAAAAEivAEAAAAAASK8AQAAAAABIrwBAAAAAAEivAEAAAAAAQq2so/NrN/JemfSnJJJyXd7+4r7RgYAAAo9fhTp3Xs+CmdWcho93hCDx6d1X1HZrZ7WAAA1NT0GU8zm5H0i5Lm3P3VkiKSfrpdAwMAAFc9/tRpPfTYSZ1eyMglnV7I6KHHTurxp05v99AAAKip1am2UUkJM4tKGpZ0pvUhAQCArY4dP6VMNl+yLZPN69jxU9s0IgAA6td0eLr7aUn/TtILkl6SdMnd/3Tr88zsATM7YWYn5ufnmx8pAAB97MxCpqHtAAB0k1am2u6QdK+kg5J2S0qa2c9ufZ67P+zuc+4+Nz093fxIAQDoY7vHEw1tBwCgm7Qy1fbvSvqhu8+7e1bSY5Je255hAQCAzR48OqtELFKyLRGL6MGjs9s0IgAA6tfKqrYvSLrLzIYlZSS9UdKJtowKAACU2Fi9llVtAQBh1HR4uvs3zOwzkr4pKSfpKUkPt2tgAACg1H1HZghNAEAotXQfT3f/oKQPtmksAAAAAIAe1OrtVAAAAAAAqIrwBAAAAAAEivAEAAAAAASK8AQAAAAABIrwBAAAAAAEivAEAAAAAASK8AQAAAAABIrwBAAAAAAEivAEAAAAAASK8AQAAAAABIrwBAAAAAAEivAEAAAAAASK8AQAAAAABIrwBAAAAAAEivAEAAAAAASK8AQAAAAABIrwBAAAAAAEivAEAAAAAASK8AQAAAAABIrwBAAAAAAEqqXwNLNxM/uMmX3PzJ4xs7/VroEBAAAAAHpDtMV//1FJX3D3d5hZXNJwG8YEAAAAAOghTYenmY1K+tuS/rEkufuapLX2DAsAAAAA0CtamWp7g6R5Sb9nZk+Z2SfMLLn1SWb2gJmdMLMT8/PzLbwcAAAAACCMWgnPqKQ7JP2Oux+RtCTpA1uf5O4Pu/ucu89NT0+38HIAAAAAgDBq5RrPFyW96O7fWP/4MyoTngAAAACABiwvS+fOSefPl/6dz0vvfe92j64pTYenu/+1mf3YzGbd/ZSkN0r6bvuGBgAAAAAh5l45Iis9Pn9eymTK7298vP/Cc917JH16fUXb5yTd3/qQAAAAAKDLuEtLS41H5MpK5X1OTEiTk9LUlLR3r3T77cXHG9s2P56cLD4/pFoKT3d/WtJcm8YCAAAAAMFzl9LpxiNydbX8/sxKI3L/fumOO6pH5I4dUrTV84Dh0T9fKQAAAIDe4y4tLjYekWsV7gRpVgzDjUg8eFCam6sckVNTxSmwkUhnv+6QITwBAAAAdAd36fLl6sFYbls2W35/AwOlEXnokHTnnbUjcqCVm3+gHMITAAAAQPu5S5cuNR6RuVz5/UUipRF5003SXXeVn8a6sW1sjIjsEoQnAAAAgOoKhcYj8sKF6hG5ORhnZ6XXvrZ6RI6OEpEhRngCAAAA/aRQkBYWGo/IfL78/qLR0mC8+ebKC+psjkizzn7d2FaEJwAAABBWhYJ08WLjEVkolN9fLFYajLfcUjsiR0aISNREeAIAAADdIJ9vLiLdy+8vHi8NxltvrR2RqRQRiUAQngAAAEC75fPFKGwkIi9erByRg4OlwXj4cOVVWTceJ5NEJLoG4QkAAABUk8s1F5GVDA2VRuK+fbUjcniYiESoEZ4AAADoH9ls4xG5sFB5f4lEaSQeOFB5GuvmiAT6DOEJAACAcMpmqwdjuaC8dKny/oaHSyPxhhuqR+TkJBEJ1InwBAAAwPZbW2s8Ii9frry/ZLI0Em+8sXZEJhKd+3qBPkN4AgAAoL1WVxuPyMXFyvsbGSkNxle8onZEDg117usFUBPhCQAAgMpWVhqPyHS68v5GR0tD8ZWvrB2Rg4Od+3oBBILwBAAA6BeZTP0L6mz8vbRUeX9jY1dD8brrpFe9qvKCOhsRGY937usF0DUITwAAgDBaXm48IpeXK+9vfPxqJO7aJb361dUjcmKCiARQN8ITAABgO7k3F5GZTOV97thxNRJ375Zuu612RMZinfua0bDHnzqtY8dP6cxCRrvHE3rw6KzuOzKz3cMC6kZ4AgAAtIt7cWpqoxG5slJ5nxMTVyNx717p9ttrR2SUX/F6yeNPndZDj51UJpuXJJ1eyOihx05KEvGJ0OCnEgAAQDnuxUVyGo3I1dXy+zMrjcj9+6U77qgekTt2EJHQseOnrkTnhkw2r2PHTxGeCA1+kgEAgN7nXrxdR6MRubZWfn9mVxfLmZqSDh6U5uZqR2Qk0tmvGz3hzEL5adWVtgPdqOXwNLOIpBOSTrv7W1sfEgAAQBXu0uXLjUdkNlt+fwMDpRF56JB0552VI3JqqrgQz8BAZ79u9K3d4wmdLhOZu8cT2zCa3tFt181m81ktZZeUXktraa34d3otfWVbei0tk+n+I/dv2xhb0Y4znu+V9Iyk0TbsCwAA9BN36dKlxiMylyu/v0ikNCJvukm6667qETk2RkSiqz14dLbkGk9JSsQievDo7DaOKtxauW42V8hVDMOq27PVn7OWrzDDYpOJxER/hqeZ7ZH09yT9uqRfasuIAABAOBUKjUfk+fPVI3JzJM7OVp7GurFtdJSIRM/ZCKFuOjsXBhuBWC4M3/+5J3S+cFmFSEZuK3Kt6IJW9Av/K6v/8dxY1ahczVe4jruMARtQKp668icZSyoVT2lqeEr7x/cXt8dSSsaT1zwnFS+/PazM3Zv/x2afkfRhSSOS3lduqq2ZPSDpAUnat2/fa55//vmmXw8AAHRIoSAtLNQfkefOSRcuSPl8+f1Fo9WDsVJEmnX26wbQcflCvuYU04rbqzxnJVdlteitfECmIQ1oSDdMTlaMvnrCcPP2wcigrM9+jpnZk+4+t3V702c8zeytks66+5Nm9vpKz3P3hyU9LElzc3PNVy4AAGhOoSBdvNh4RBYK5fcXi5UG4y231I7IkREiEgi5jUAsF4GtTDFtJBAHbKBs4E0kJrR3dG/dcfgLn/qOzi2azIdkGpIpLpNpZjyhr73nDQF+F/tXK1Nt75b0NjN7i6QhSaNm9t/c/WfbMzQAAHCNfL7xiLx4sXJExuOlwXjrrbUjMpUiIoEuVvDClcir+/rDtbTS2erPyeTqX0XXZGWjb3xoXHtG9xS3x+o7a7h5+1B0qC1nEH/tzXu4brbDmg5Pd39I0kOStH7G831EJwAADcjni2cWG43ISpfJDA6WBuPhw5UX1Nl4nEwSkcA2KXhBy9nl+sOwzmmmjQbiRuBtjr6xoTHNjM40Nc30y9+9pI/+7+f10qWV4vWob+i+61HDdt1st63A2wzu4wkAQDvkcpUjslJQLixUjsihodJIPHKkdkQODxORQAA2ArGd1x+m19Jazi43NI5y0Tc6OKrrU9c3df1hKp5SIppo6zWIjz91Wr/22WebWi220+47MtN1YyqnlRV4u0lbwtPdvyLpK+3YFwAA2y6bbS4iK0kkSiNx//76IjKkeuGdebRmu44Bd79yBrGdU0yXsksNjSMZS5aNvl2pXU1PMU3EEhqw7l+x+djxUyXTVyUpk83r2PFT/BxoUq98TznjCQDobdns1dt21BuRly5V3t/wcGkkHjxYPSInJ0MdkY3qlXfm0bx6joGNQGwoDiucTdz8+eXsslz1r2U5HBsuG307UzubXsl0ODYcikAMypmF8tN8K21Hbb3yPSU8AQDhsbZ2bSjWisjLlyvvL5ksjcRDh6rf7mNysnj2EhWF7Z15zs5W5+7K5DINheGnnjiltBbl8VUVlJFrRQVb0Ts/u6qxL11dFbWRQExEE2Wjb3p4uukppv0eiEHZPZ7Q6TJBtHucn53N6pXvKeEJANgeq6uNR+TiYuX9pVKlkXjTTbUjcmioc19vnwjTO/O9dHbW3bWSW2nsjGEdt7loJhBX8zHZQEIDKt6mYsCHFPUxDeQGde/sbMNTTIdjw4oMRAL87qGdHjw6y2qxbdYr31PCEwDQupWVxiMyna68v9HR0mCcna0dkYODnft6UVGY3pnfjrOzG4HYjimmm5+zlF1SwSvcMqeMoehQ2fCbHJ4sPo7VjsOt25KxpCIDEd39kS+XPQZmxhP6Lz/F/RF7XdhWiw2DXvmeEp4AgFIrK/Xd2mPztqUqC2+MjV0Nxelp6eaba0dkPN65rxdtFaZ35qudnXV3reZX23qLi43tjQbiRuAVCkO6mDblcnElYim9+vr9euWB6xqeZroRiEEJ0zGAYIRltdgw6YXvKeEJAL0sk2k8IperLO8/Pn41FHfulG65pXpETkwQkX2m0+/MbwRiM3G4mHxOS9kluVbktqKCVuSWkWxVsQ+tKO/52gNYNxgZLBt9e0b3NHX94caqqNGB4q9qG9OCBzZibk2afz6iX3rNrV33y2ivnJ0B0F7mle4fFoC5uTk/ceJEx14PAHrK8nJ9Ebn5cabKdXU7dlQOxq2PNyIyFuvc14ue4u5ay6+1/TYX6bV0Q4EYj8SvxF0hP6hziyYvDGlAgzJPKDYwrJ+8cUaHZ3Y1FIwbgRiUatNXv/YBpq8C6B5m9qS7z23dzhlPAOg09/ojcvO2lZXK+5yYuBqJe/ZIt99eOyKj/C8A5W0EYrunmeYKubrHEI/Ey0bf7pHdTd3iYmNbLFL65klYVrUN06JNAFAOv3UAQCvci9c3NhqRq6vl92dWGpH79klHjlSPyB07iMg+tZZfazwM61jJtJFAjA5ENRIfuSb6rk9d39IU03ikM1O0w3LdVJgWbQKAcvhNBQA2uBdXWm10OuvaWvn9mV1dLGdqSjpwQHrNa2pHZITbBvSabD5b9ab3zZ5JzBaydY8hOhAtG307Uzt1KH6orlVMy23vVCD2OxbsARB2hCeA3uRevOdjoxGZrfCL/MBAaUQeOiTdeWf1iBwfJyJDJpvPtv02F40GYsQiGhkcuSbwrktepxt23HBle6NnEgnEcGPBHgBhx+JCgMJzjU/fcpcuX24sIs+frxyRkUhpRFZblXXj8dhYMT7RFXKFXNvCcH7pkhYyi8p5RrL6p5hGLFI7/JqYZhqPxGVmAX73AAAIDosLARVsLFG/MX3p9EJGDz12UpK6Lj57IpDdpUuXGo/IXIUgiERKI3F2tnZEjo4SkR2SK+RKbnDfrpVMV/MVrpEtY8AGroTd5uibGp5S1K/TmaU1JQqDMh+SaUiDA8P6B3cc0k8e2ls1GAlEAADqR3ii7x07fqrkmhlJymTzOnb8VFdFXVcGcqFQPiKrBeX581K+wq0PotHSSLz55tpnJUdHi9dSoiX5Qj6QKabNBOLWs4GTw5PaH9/f9JnEwchgxUC8+yNf1vjqtQu2fOt7CX3sPm5RAQBAuxCe6HthWaI+8EAuFKSFhcYjslAov79YrDQSb7mldkSOjGxLRIbpTPJGILbzFhfptbRWclVu1bKFycpG346hHdo7urfpKaZD0aGOn0EMy3//AACEHeGJvheWJeob+gU5n288Ii9cqByR8XhpMN56a+2ITKVCcSYyqDPJ+UJey9nltt/motFALBd940PjmhmdKW5vYiXT7QjEoITlv38AAMKO8ETf6/ol6vN56eJF3bU2r+zZs9qRWdSO5cuayFzWeOay9uSXpXsfvjYiKy0cNjhYGoyHD9eOyGQyFBHZjN/8wjNayi7KtaqCZeRa0Up+Rb/8+W8pP3hj02cTM7nGzpiVi76xoTHNjM40PcU0EU30TCAGpev/+wcAoEcQnuh7HV2iPpcrRmG1BXW2brt4UXLXo2V2txKNqzA5KeV3FkPxyJHKC+psPB4eDmVEFrxw5QxiO6eZLmeXpTInt17OSu/4w2u3X7mVxaboG4mP6PrU9U1PMU3EEhqw3lrsKCzTl7lFBQAAncHtVIBm5XJXr3NsJCIrSSRq3trjawvSf33msn6Qi2tw10794k8d7rpfkAteUCabafsU0+XsckPjGI4NV4++9Smm//0v55VeiWpgfUXTjb+vS43r0+9+Xcm/7cVADMLW6ctS8Szih99+a9cdrwAAoL24nQpQTTbbeEQuLFTe3/BwaUQePFj9TOTkZPHf1HD3+p92cHctZ5fbvpLpUnapoXFsBOLWONyZ2tn0PRGHY8N1B+JPTK9HUq40kj705lt1287ui6QwnEkMy0rRAACgc5oOTzPbK+lTknZJKkh62N0/2q6BAU1bW2s8Ii9dqry/ZLI0Eg8dqh2RifYtTOLuyuQybY3DjTOIrvpnPCSiibLRNz083dT1h6l4qqFADEqYplp25S11ymClWAAAsFUrZzxzkv61u3/TzEYkPWlmX3T377ZpbIC0utp4RF6+XHl/qVRpJN50U9WprZqclIaG6hrqRiAuraWVvvjXbbvNxdLaUsOBWC76poeni9tjjV1/uBGIkYFI3WMIm/uOzHRVuFUSljOJrBQLAAC2ajo83f0lSS+tP140s2ckzUgKZXiGYfpa6K2uVg/GckG5uFh5fyMjpZE4O1s7IgcH5e5aya1Uib6XtJR9VukX0ko/u2l7tvbZxkYCcSg6VDb6Jocnm55imowlezoQ+11YziSyUiwAANiqLdd4mtkBSUckfaPM5x6Q9IAk7du3rx0v13Zhmb7WVVZWakbky8+d1sUfv6TU4oImVhY1vFbll+OxsauhOD0t3XyzfGpSqxNjSk+OKD2W0NJYQumRQaWHo1pKRJQurFY4S/is0mtPF7f/KK309699TsEr3K+yjMHIYNnom0hMtDTFNDrAJdZoTFjOJIZp+jIAAOiMlle1NbOUpD+T9Ovu/li153brqrZ3f+TLZX+ZmxlP6GsfeMM2jKjDMpnGz0QuXV1AxiWtRqV0XFqKSenJEZ0dS+rZaFQvJ4Z0LhnXhURcl4ejmjm4QyM7k1efG8kprTUt5cqvgtpoIFaNviammCbjSQIRXYPVYgEAQLcLZFVbM4tJ+iNJn64Vnd0sLNPX6rK8XDEi/fw5rZ0/q/TFl5W+fF5Li+eVXrqotK8WIzC+Ho/xTY9Tg0qPxJU+FNPSLRGlh0zpWFJLkYTSA+vR6KvK++brzhbX/5QXvxgvG30zozMtTTGNRWKBf3uB7cSZRAAAEFatrGprkn5X0jPu/h/aN6TO67bpa+6utdyqli7NK/3yj5U+d0ZL518qBuPCWS1dXg/GpYtaWrm8fv3hkpbyK0pH8lfPJm4NyQkpP1X/OOKRuJKb74W4Hn4zdcThez79XZmGZJ7QgIZkPqQBFf/86CP3BvfNA3pcWBZCAgAA2KyVM553S/o5SSfN7On1bb/s7p9rfVid1cpCGGv5tTpucbGodPqili6fU3r9LONS5pLSK4vFBWtyGaULK8Wzh5ZTOppXrtodJgYkjaz/kRR100ghpqQNKxUZUiqaVDKe1PVDo0oNjyuZ3KHUyIRSg6N1TzNNxpOKR+JNf08/Olp5+jIAAACA/tLKqrZ/LsnaOJZtc9+RGV1YeVn/9kuP6OLKopJDWR3al9CfvPCHevT/lYnKlUWl1xa1lF1W1nN1v040L6XWin+S2auPdyqmQzakVHSsGIODI0olxpRMjiuVmlRqbErJsWmlJnYpNbVbycldSg2NXQnGVgIxKKxqCQAAAGADq6asu2XHBf0w/9tSTFrMD+jPn4spVYgqlRtQcs2VWnHtXM4puZxVaqVQjMe1LSGZNaUGR5QcHlMqNaHUyJSSY1NK7dip1MQuxad3XXu7j/FxKVL/7S8ef+q0jv3eKZ1ZeL6rr+/iWjQAAAAAGwjPdXecH9S53yhGZDxfkEVy0uTYtfeCPFjm/pCbI3Kg2hzZ1oTtti9ci9bfuDcuAAAANhCe62KHj2jyy1+/GpSjo4FGZDOOHT9VMnVVkjLZvI4dP8Uv9OgqYXuTBAAAAMHqrrLaTqmUdNdd0o03Bn7mslk9ddsX9LRqb5IAAACg/3RfXaGiSrd32a7bvgCV8CYJAAAANiM8Q+TBo7NKxEoXImKlWHQj3iQBAADAZoRniNx3ZEYffvutmhlPyFS8J+aH334r18yh6/AmCQAAADZjcaGQYaVYhAG30wEAAMBmhCeAQPAmCQAAADYw1RYAAAAAECjCEwAAAAAQKMITAAAAABAowhMAAAAAECjCEwAAAAAQKHP3zr2Y2byk5zv2gs2ZknRuuweBvsdxiG7AcYhuwbGIbsBxiG4QhuNwv7tPb93Y0fAMAzM74e5z2z0O9DeOQ3QDjkN0C45FdAOOQ3SDMB+HTLUFAAAAAASK8AQAAAAABIrwvNbD2z0AQByH6A4ch+gWHIvoBhyH6AahPQ65xhMAAAAAECjOeAIAAAAAAkV4AgAAAAAC1ZfhaWb3mNkpM3vWzD5Q5vNmZv95/fPfNrM7tmOc6G11HIc/s378fdvM/sLMDm/HONH7ah2Lm573N8wsb2bv6OT40B/qOQ7N7PVm9rSZ/ZWZ/Vmnx4jeV8f/m8fM7E/M7Fvrx+H92zFO9DYze8TMzprZdyp8PpSt0nfhaWYRSb8t6c2SXiXpnWb2qi1Pe7Okm9b/PCDpdzo6SPS8Oo/DH0p6nbvfJulDCvHF5OhedR6LG8/7DUnHOztC9IN6jkMzG5f0MUlvc/dbJP3Djg8UPa3On4f/QtJ33f2wpNdL+vdmFu/oQNEPPinpniqfD2Wr9F14SrpT0rPu/py7r0n6A0n3bnnOvZI+5UVPSBo3s+s7PVD0tJrHobv/hX7f4nMAAAKsSURBVLtfXP/wCUl7OjxG9Id6fiZK0nsk/ZGks50cHPpGPcfhP5L0mLu/IEnuzrGIdqvnOHRJI2ZmklKSLkjKdXaY6HXu/lUVj61KQtkq/RieM5J+vOnjF9e3NfocoBWNHmPvlvT5QEeEflXzWDSzGUl/X9LHOzgu9Jd6fia+QtIOM/uKmT1pZj/fsdGhX9RzHP6WpJslnZF0UtJ73b3QmeEBV4SyVaLbPYBtYGW2bb2nTD3PAVpR9zFmZn9HxfD8iUBHhH5Vz7H4nyS9393zxTf5gbar5ziMSnqNpDdKSkj6upk94e7fD3pw6Bv1HIdHJT0t6Q2SDkn6opn9H3e/HPTggE1C2Sr9GJ4vStq76eM9Kr5r1ehzgFbUdYyZ2W2SPiHpze5+vkNjQ3+p51ick/QH69E5JektZpZz98c7M0T0gXr/33zO3ZckLZnZVyUdlkR4ol3qOQ7vl/QRd3dJz5rZDyW9UtJfdmaIgKSQtko/TrX9v5JuMrOD6xeD/7Skz255zmcl/fz6ilF3Sbrk7i91eqDoaTWPQzPbJ+kxST/HO/oIUM1j0d0PuvsBdz8g6TOS/jnRiTar5//NfyzpJ80sambDkv6mpGc6PE70tnqOwxdUPOsuM9spaVbScx0dJRDSVum7M57unjOzf6niyowRSY+4+1+Z2T9b//zHJX1O0lskPStpWcV3t4C2qfM4/FVJk5I+tn6mKefuc9s1ZvSmOo9FIFD1HIfu/oyZfUHStyUVJH3C3cveagBoRp0/Dz8k6ZNmdlLF6Y7vd/dz2zZo9CQze1TFVZOnzOxFSR+UFJPC3SpWnCkAAAAAAEAw+nGqLQAAAACggwhPAAAAAECgCE8AAAAAQKAITwAAAABAoAhPAAAAAECgCE8AAAAAQKAITwAAAABAoP4/srXOgQKqlNEAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "from sklearn.linear_model import LinearRegression\n", "from skbonus.linear_model import LADRegression\n", "\n", "x = np.array([0, 1]).reshape(-1, 1)\n", "plt.figure(figsize=(16, 4))\n", "plt.scatter(X, y)\n", "plt.plot(x, LinearRegression().fit(X, y).predict(x), 'r')\n", "plt.plot(x, LADRegression().fit(X, y).predict(x), 'g')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### See also\n", "scikit-learn tackles this problem by offering a variety of [robust regressors](https://scikit-learn.org/stable/auto_examples/linear_model/plot_robust_fit.html). Many of them use an indirect approach to reduce the effect of outliers. [RANSAC](https://en.wikipedia.org/wiki/Random_sample_consensus), for example, samples random points from the dataset until it consists of all inliers.\n", "The closest thing to LADRegression that scikit-learn offers is the [HuberRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.HuberRegressor.html) with a loss function that is partly a squared and partly an absolute error. However, it is more complicated and requires hyperparameter tuning to unleash its full potential.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.7" } }, "nbformat": 4, "nbformat_minor": 4 }