diff --git a/lecture_4.ipynb b/lecture_4.ipynb
deleted file mode 100644
index 9e82ed2664ff3139d957d3ac61544a3d99e7186a..0000000000000000000000000000000000000000
--- a/lecture_4.ipynb
+++ /dev/null
@@ -1,557 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "id": "d2193805-8359-4380-82da-4b1cb5358290",
- "metadata": {
- "slideshow": {
- "slide_type": "slide"
- },
- "tags": []
- },
- "source": [
- "# Lecture 1\n",
- "\n",
- "---\n",
- "\n",
- "## Basic statistics \n",
- "\n",
- "<br>\n",
- "<br>\n",
- "\n",
- " Hartmut Stadie\n",
- "\n",
- "hartmut.stadie@uni-hamburg.de"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "668ad170-00ca-4d67-ba06-71aebd7092a3",
- "metadata": {
- "jp-MarkdownHeadingCollapsed": true,
- "slideshow": {
- "slide_type": "slide"
- },
- "tags": []
- },
- "source": [
- "# Parameterschätzung\n",
- "\n",
- "## Einführung\n",
- "\n",
- "### Schätzer \n",
- "\n",
- "Schätzer $\\hat a$ für $a$ aus Stichprobe $x_1,\\dots x_n$\n",
- "\n",
- "Anforderungen:\n",
- "\n",
- "- erwartungstreu: \n",
- " $E[\\hat a]= a$\n",
- "\n",
- "- konsistent: \n",
- " $\\lim_{n\\to \\infty } \\hat a = a$\n",
- "\n",
- "- effizient: $V[\\hat a]$ möglichst klein\n",
- "\n",
- "Schätzer für den Mittelwert:\n",
- "$$\\hat \\mu = \\bar x = \\frac{1}{n}\\sum_1^n x_i \\text{ mit } V[\\hat \\mu] = \\frac{\\sigma_x^2}{N}$$"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "7c7c6ecd-1611-4787-8985-da0881e90d9f",
- "metadata": {
- "slideshow": {
- "slide_type": "slide"
- },
- "tags": []
- },
- "source": [
- "# Methode der kleinsten Quadrate\n",
- "\n",
- "## Herleitung\n",
- "\n",
- "### Methode der kleinsten Quadrate \n",
- "\n",
- "$y(x) = mx + a$: Finde $\\hat m$ und $\\hat a$!"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "73e6e46a-4e41-4e66-857b-2946946498d1",
- "metadata": {
- "slideshow": {
- "slide_type": "skip"
- },
- "tags": []
- },
- "source": [
- "<img src=\"./figures/11/line.png\" style=\"width:90.0%\" alt=\"image\" />"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 1,
- "id": "55bf0cad-8a50-4831-8a37-daf9bcb39b71",
- "metadata": {
- "slideshow": {
- "slide_type": ""
- },
- "tags": []
- },
- "outputs": [
- {
- "data": {
- "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjMAAAG0CAYAAADHD6Y/AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAA3VElEQVR4nO3deXgV5eH28e9JQhKCSQDZCwq4gIKoBVQURQVxQRGlra+11uXXVhBUsIugrYq1oG1FWhco1lqtdWlFEEVlUQF3AVHBBTcQFBBRTJBAQpJ5/5gaRNkSksyZk+/nus7V80xmknuuocntM8tJBEEQIEmSFFNpUQeQJEnaHZYZSZIUa5YZSZIUa5YZSZIUa5YZSZIUa5YZSZIUa5YZSZIUa5YZSZIUa5YZSZIUa5YZSZIUa5GWmblz53LaaafRqlUrEokEU6ZM2errQRBw7bXX0qpVK+rXr8+xxx7Lm2++GU1YSZKUlDKi/OEbNmzg4IMP5oILLmDgwIHf+fof//hHxo4dyz//+U/2339/rr/+ek444QSWLFlCbm7uLv2M8vJyVq5cSW5uLolEorp3QZIk1YAgCFi/fj2tWrUiLW0ncy9BkgCCyZMnV4zLy8uDFi1aBDfccEPFsk2bNgX5+fnBhAkTdvn7rlixIgB8+fLly5cvXzF8rVixYqd/6yOdmdmRpUuXsnr1avr27VuxLCsri169evHCCy9w0UUXbXO74uJiiouLK8bB/z4UfMWKFeTl5dVsaEmSVC0KCwtp06bNLp2JSdoys3r1agCaN2++1fLmzZvz0UcfbXe7MWPGMGrUqO8sz8vLs8xIkhQzu3KJSNLfzfTtnQiCYIc7NnLkSAoKCipeK1asqOmIkiQpQkk7M9OiRQsgnKFp2bJlxfI1a9Z8Z7bmm7KyssjKyqrxfJIkKTkk7cxMu3btaNGiBTNnzqxYVlJSwpw5czjyyCMjTCZJkpJJpDMzX331Fe+//37FeOnSpbz22ms0btyYvfbai2HDhjF69Gj2228/9ttvP0aPHk1OTg4//vGPI0wtSZKSSaRlZv78+Rx33HEV48svvxyA8847j3/+85/85je/YePGjVx88cWsW7eOww8/nBkzZuzyM2YkSVLqSwRf37ucogoLC8nPz6egoMC7mSRJionK/P1O2mtmJEmSdoVlRpIkxZplRpIkxZplRpIkxZplRpIkxZplRpIkxZplRpIkxZplRpIkxZplRpIkVVpRSSltR0yj7YhpFJWURprFMiNJkmLNMiNJkmLNMiNJkmLNMiNJkmLNMiNJkmLNMiNJkmLNMiNJkmLNMiNJkmLNMiNJkmLNMiNJkmLNMiNJkmLNMiNJknbL6oJNkf58y4wkSaq0SQs+rnjfZ+wcHpy3PLIslhlJklQpqwo2cs3UNyvG5QFc+fBiVhVsjCSPZUaSJFXK0rUbKA+2XlYWBCxbWxRJHsuMJEmqlHZNGpCW2HpZeiJB2yY5keSxzEiSpEppmV+fUf07VYzTEjD6zM60zK8fSR7LjCRJqrSBXVtXvJ91eS/O6r5XZFksM5Ikabe0yM+O9OdbZiRJUqxZZiRJUqxZZiRJUpV0SCwnk81Rx7DMSJKkSirbTMZzf+axzKsYljEp6jRkRB1AkiTFzFOjyHzhFkjAXolPISiPNI4zM5IkqXJ6DKW8UXsuLRnC0M2XQiLaOmGZkSRJO7b2fXh27JZxbgs2XfQiU8uPAhLb3ay2eJpJkiRtW3k5vPI3mDUKSjdC0w7QsV/4tbTkqRDJk0SSJCWPdcvgkaGw7Nlw3P5YaNElykTbZZmRJElbBAHM/wfM+B1s3gD1GkDf66Db/0Ei+lNK22KZkSRJW0wZDK/fH77f60gYcDs0bved1XIyM1h2Q79aDrdtXgAsSZK2OHAAZGTDiWPg/GnbLDLJxpkZSZLqsvWr4bMl0L5XOO5wElz2BuQ2jzZXJTgzI0lSXRQEsOghuO1w+M+5ULhyy9diVGTAmRlJkuqeDWvhseHw9tRw3PJg2Lwx2ky7wTIjSVJd8tYj8NjlULQ2fFbMMb+Boy+H9HpRJ6syy4wkSXVBeTlMvggW/SccN+sEZ4wPZ2VizjIjSVJdkJYGOY3Dz1HqORx6XQEZWVGnqhaWGUmSUtWmAijZAHmtwnHva6DLWfC970ebq5p5N5MkSanog6fh9iPhof8LTzEBZOakXJEBZ2YkSUotxV/BzKth/p3hOL0erF8J+a2jzVWDLDOSJKWKZc/BlIvhy4/C8WG/gD7XQmaDSGPVNMuMJElxt3kTzLoWXh4fjvPbwOm3bXmqb4qzzEiSFHeJNFj2bPj++z+Fvn+A7LxoM9Uiy4wkSXG0eROkpYfXxGRkwhkTws9Z2u+EqJPVOu9mkiQpblYuhIm94NmxW5a1OKhOFhmwzEiSFB+lJfDMaLijN3z2Drx6N5QURZ0qcp5mkiQpDlYvhimDYPWicNzpDDjlpvDZMXWcZUaSpGRWVgrP3wyzb4TyzVC/MfS7CTqfGXWypGGZkSSpFhWVlHLg1dMBeOu6E8nJ3Mmf4oIVMPemsMh06Aen3gy5zWshaXxYZiRJSjZBAIlE+L5xOzj5BsjIDj9X6evlquAFwJIkJZPPP4B/9oPlL21Z1vV8OPj/WWS2wzIjSVIyKC+HlyfChJ7w0fPw+K/DGRrtlKeZJEmK2pfL4ZEhsHRuOG57dPhxBM7E7BLLjCRJUQkCWPBPmH4VlHwF9XLghOug2/9BmidPdpVlRpKkiKR9+BQ8elk4aHMEDLgd9twn2lAxZJmRJCki5e17wwH9oc3hcMTg8LOWVGlJPYdVWlrKb3/7W9q1a0f9+vVp37491113HeXl5VFHkySp8tZ/Subjw8hjQzhOJOBH98CRQy0yuyGpZ2ZuvPFGJkyYwN13302nTp2YP38+F1xwAfn5+Vx22WVRx5Mkadctfhim/ZKMjV9wVcYKrij9Rbjci3x3W1KXmRdffJHTTz+dfv36AdC2bVvuv/9+5s+fH3EySZJ20YbP4fFfwpuTAShvfhB3LT8p4lCpJalPM/Xs2ZOnnnqKd999F4DXX3+d5557jlNOOSXiZJIk7YJ3psHth4dFJpEOva5g0/kzeCfYK+pkKSWpZ2auuOIKCgoK6NixI+np6ZSVlfGHP/yBs88+e7vbFBcXU1xcXDEuLCysjaiSJG1t3p0w7fLwfdMD4Izx0OpQKCmtWGV1wSbaN90jooCpI6lnZh588EHuvfde7rvvPl599VXuvvtu/vznP3P33Xdvd5sxY8aQn59f8WrTpk0tJpYk6X86nQG5reCoYXDRnLDIAJMWfFyxSp+xc3hw3vKIAqaORBAk77OS27Rpw4gRIxgyZEjFsuuvv557772Xd955Z5vbbGtmpk2bNhQUFJCXl1fjmSVJddSmQlj0n/CBd19f1Fv8FWRtmXlZVbCRo254mvJv/OVNTyR4bsRxtMyvX8uBk1thYSH5+fm79Pc7qU8zFRUVkfatJyCmp6fv8NbsrKwssrKyajqaJKmWFZWUcuDV0wF467oTyclMoj9hH84JP46gYAVk5UOXH4bLs7Y+hbR07YatigxAWRCwbG2RZWY3JNG/hO867bTT+MMf/sBee+1Fp06dWLhwIWPHjuXCCy+MOpokSVCyAWZdC69MDMcN94a8VttdvV2TBqQl+M7MTNsmOTWbM8UldZm55ZZb+N3vfsfFF1/MmjVraNWqFRdddBFXX3111NEkSXXdRy/ClMGwbmk47nYhnPD778zGfFPL/PqM6t+J3z3yJgBpCRh9ZmdnZXZTUpeZ3Nxcxo0bx7hx46KOIknSFs+NC2dkCCDve9D/Fti39y5tOrBr64oyM+vyXt7NVA2SusxIkpSUWh0KBHDIT+Ck0ZCdX6Vv0yI/u3pz1VGWGUmSdqa0GFYvhtZdw3H7XnDxS9DsgGhzCUjy58xIkhS5Va/DxOPg7tPgi6VblltkkoZlRpKkbSnbDLNvhDuOhzVvQr36UPDxzrdTrfM0kyRJ37bmbZh8UTgrA3BAfzj1ZmjQJNpc2ibLjCRJ3/TCLfDUdVBWAtkNod9N0Hnglqf6KulYZiRJ+qaiz8Mis/9JcNpfILdF1Im0E5YZSVLdVl4OG7/Ycgrp2JHQokv4QZHOxsSCFwBLkmJndcGm6vlGXyyFu0+Ff/8AykrDZRlZ0PlMi0yMWGYkSbEwacGWO4n6jJ3Dg/OWV/2bBQHM+zuMPwo+eh4+exc+XVQNKRUFy4wkKemtKtjINVPfrBiXB3Dlw4tZVbCx8t/syxXwrwEw7ZeweQPs3RMGP/+/p/oqjrxmRpKU9Jau3bDVJ00DlAUBy9YW7fqHNAYBvPZveHIkFBdCRjb0uRYOuwjS/G/7OLPMSJKSXrsmDUhLsFWhSU8kaNskZ9e/SVAO8+8Ki0zr7jBgAjTZt/rD7kROZgbLbuhX6z83lVlFJUlJr2V+fUb171QxTkvA6DM773xWJgigvOx/G6XDgPHQZxRcOD2SIqOaYZmRJMXCwK6tK97PurwXZ3Xfa8cbfPUZ/OdceGrUlmVN94eew8Jio5ThaSZJUuy0yM/e8QpvPQKPDQ8fgJeeBUdc7MPvUphlRpKUOoq+gMd/DYsfCsfNOsEZ4y0yKc4yI0lKDUuehEcvha8+hUQa9BwOva4IH4KnlGaZkaQUV1RSyoFXTwfgretOJCczBX/1F30Bk34GJeuhyf7hnUqtu0adSrUkBf9FS5LqnJzGcNIY+OwdOP63UG8Xnz2jlGCZkSTFT/F6mHEdHHAa7HN8uOz750abSZHx1mxJUqwcnnib7L8fA/P/AVMvhdLiqCMpYs7MSJLiYXMR12TczQUZ06EAyN8LTr/VC3xlmZEkxcDyl8meMpgLMj4AoPSQn5Jx0h8gOy/iYEoGnmaSJCW3T9+Eu04i7YsPWBU05rySKyg55WaLjCo4MyNJSm7NO8EB/SlNz+LEeb0pZI+oEynJODMjSUoupSUw98+wYe2WZQP/Tslpt1tktE3OzEiSksfqxTBlEKxeBKtehx/dA4kEpNcjJx2W3dAv6oRKQpYZSVL0ykrh+XEw+wYo3wz1G0OnM8IiI+2EZUaSFK3PlsDkQbDy1XDcoR+cNg72aBZpLMWHZUaS6pDVBZto3zSJrjt5fxbc/2MoK4bsfDj5j9DlLGdkVCleACxJKW7Sgo8r3vcZO4cH5y2PMM23tO4ODZrCvn3g4pfg4P9nkVGlWWYkKYWtKtjINVPfrBiXB3Dlw4tZVbAxmkDl5fD2oxAE4Tg7H342E855CPJaRZNJsWeZkaQUtnTtBsqDrZeVBQHL1hbVfph1H8E9/eHBn8Crd29ZntfK2RjtFq+ZkaQU1q5JA9ISbFVo0hMJ2jbJqb0QQRCWl+lXQclXUC8HsLyo+jgzI0kprGV+fUb171QxTkvA6DM70zK/fu0EKPgE7h0Ij14WFpm9esCg56DrebXz81UnWGYkKcUN7Nq64v2sy3txVve9aucHvzMNbu8BHzwF6VnQ93o4fxrsuU/t/HzVGZ5mkqQ6pEV+du39sAbNoGQ9fK8rDJgATfevvZ+tOsUyI0mqPp9/sGXmpU13+OkjsNeRkO6fG9UcTzNJknbfhs/hP+eFp5U+W7JlebtjLDKqcf4LkyTtnrcfg8eGwYbPIC0DVrwCTTtEnUp1iGVGklQ1G9fBE1fAGw+G46YHwBnjodWh0eZSnWOZkSRV3nuzYOpQWL8KEmlw5KVw3JWQkRV1MtVBlhlJUuWtei0sMnvuG96p1KZ71IlUh1lmJEm7ZvNGqPe/h+0dNSychen2f5BZi08TlrbBu5kkSTtWsgGm/QruOB5Ki8Nl6Rlw5CUWGSUFZ2YkSdv30YswZTCsWxqO35sJB5wabSbpWywzkqTv2rwRnr4eXrwNCCCvNZx+K+xzXNTJpO+wzEhSisvJzGDZDf12fYOP58PkQfD5e+H40J/AiaMhO79mAkq7yTIjSdra078Pi8weLaD/X2H/E6NOJO2QZUaStLXT/gpz/wQnXAc5jaNOI+2UZUaS6rKyzfDsTbDxSzj5hnBZo73D62OkmLDMSFJd9elbMGUQrHo9HB/yY2jZJdpMUhVYZiSprikrhRf+CrPHQFkJZDeEfjdBi4OiTiZViWVGkuqSte+Fdyp9Mj8c738SnPYXyG0RbS5pN1hmJKmuKC2Bu/vD+pWQlQcn3RCeWkokok4m7RbLjCTVFRmZ0Pf38Nq/of8tkN866kRStbDMSFKqKi+H+XeGpaXDyeGyzgPDl7MxSiGWGUlKRV+ugEeGwNI50KApDHklfGaMJUYpyDIjSakkCGDhvfDkSChZDxn14ehfhXcsSSnKMiNJqaJwFTx6Kbw3Ixy3PgwGjIcm+0abS6phlhlJSgXrP4Xbj4BNX0J6Jhz/W+gxFNLSo04m1TjLjCSlgtzm0PFUWPMmDJgAzTpGnUiqNZYZSYqrtx6B1t0hr1U4PuWP4axMer1oc0m1LC3qAJKkSir6Ah66EP7zU5h6aXjRL0BmA4uM6qSkLzOffPIJP/nJT9hzzz3JycnhkEMOYcGCBVHHkqRoLHkyvDZm8SRIpEOrQ6C8LOpUUqSS+jTTunXrOOqoozjuuON44oknaNasGR988AENGzaMOpok1a6NX4a3W79+Xzhu0gHOGA/f6xppLCkZJHWZufHGG2nTpg133XVXxbK2bdtGF0iSorDmbbh3IBR+AiTgyEvguKugXnbUyaSkkNSnmaZOnUq3bt344Q9/SLNmzTj00EO54447oo4lSbWr4d6QkQ2N28OF08PPV7LISBWSusx8+OGHjB8/nv3224/p06czaNAgLr30Uu65557tblNcXExhYeFWL0mKnU8WhJ+tBJCZAz/+Dwx6DvY6PNpcUhJKBMHXl8Enn8zMTLp168YLL7xQsezSSy9l3rx5vPjii9vc5tprr2XUqFHfWV5QUEBeXl6NZZWkalFSBLOuhVf+BieOhh5Dok4kRaKwsJD8/Pxd+vud1DMzLVu25MADD9xq2QEHHMDy5cu3u83IkSMpKCioeK1YsaKmY0pS9Vj+MkzoGRYZCD8sUtJOJfUFwEcddRRLlizZatm7777L3nvvvd1tsrKyyMrKquloklR9Nm+CZ66HF24FAshtBaffAvv2iTqZFAtJXWaGDx/OkUceyejRo/nRj37EK6+8wsSJE5k4cWLU0SSpeqx6Ayb9DNb+7z/cDj4bTroB6jeMNJYUJ0ldZrp3787kyZMZOXIk1113He3atWPcuHGcc845UUeTpOqRSMAXH0KDZnDaX6DjKVEnkmInqS8Arg6VuYBIkmpF0ReQ03jL+O3HYO8jt14m1XEpcwGwJKWUslKY8ye4uROsfG3L8gNOtchIuyGpTzNJUspY8w5MGQQrF4bjxZPCz1WStNssM5JUk8rL4MXb4OnroawYsvPh5D9Blx9FnUxKGZYZSdqOopJSDrx6OgBvXXciOZmV/JX5+QcwZTCseDkc73sC9P8r5LWq5qRS3WaZkaSa8t7MsMhk5sJJo+HQc8O7lyRVK8uMJFWn8nJI+9+9FYf9AtavhO4/g4Z7RZtLSmHezSRJ1SEIYP5dMPEYKNkQLktLgxOus8hINcwyI0m7q+ATuHcgPDYMVi+CBf+MOpFUp3iaSZKqKgjg9fvhiRFQXAAZ2dD7ajh8cNTJpDrFMiNJVbH+U3j0Mnj3iXD8va4wYAI03T/aXFIdZJmRpKqYcVVYZNIz4diRcOSlkO6vVCkK/j9PkqrihN/DV2vCT7hufmDUaaQ6zQuAJWkXpC95DKZftWVBXks4b6pFRkoCzsxI0g7k8RWj6t1N1qTnwwX7HA/79o42lKStWGYkaTvSPpjFjKwraJFYR5BII3HUMGjbM+pYkr7FMiNJ37apEKZfSfbCfxHQmIdKe9LtrCtpe9BRUSeTtA2WGUn6piCAfw2ATxbwQOmxjCz9GQFppN33JWPOXM5Z3X2ar5RsvABYkr4pkYCjf8WqvIO5suwXBP/7NVkewJUPL2ZVwcaIA0r6NsuMJH30Arwzbcu44yksPfVByoOtVysLApatLardbJJ2ytNMkuquzRvhqd/DS7dDdh5c/HJ4yzXQrnlD0hJsVWjSEwnaNsmJKKyk7XFmRlLd9PF8mHA0vHQbEMAB/SGzQcWXW+bXZ1T/ThXjtASMPrMzLfPrRxBW0o5Uusycf/75zJ07tyaySFLNKy2GWdfCnSfA5+/BHi3gx/+F028NZ2e+YWDX1hXvZ13ey4t/pSRV6TKzfv16+vbty3777cfo0aP55JNPaiKXJFW/zRvhjuPhuZshKIcuZ8GQl2D/vjvdtEV+di0ElFQVlS4zkyZN4pNPPmHo0KH897//pW3btpx88sk89NBDbN68uSYySlL1qFcf9j4ScprAWffCmROhfqOoU0naTVW6ZmbPPffksssuY+HChbzyyivsu+++nHvuubRq1Yrhw4fz3nvvVXdOSaqaT9+Cdcu2jPtcC0NehgNOiyqRpGq2WxcAr1q1ihkzZjBjxgzS09M55ZRTePPNNznwwAO5+eabqyujJFVeWSk8exNM7AVThkB5ebg8swE0aBJtNknVqtK3Zm/evJmpU6dy1113MWPGDLp06cLw4cM555xzyM3NBeCBBx5g8ODBDB8+vNoDS9JOffYuTBkMn8wPx9l5sHkDZOVGm0tSjah0mWnZsiXl5eWcffbZvPLKKxxyyCHfWefEE0+kYcOG1RBPkiqhvAxeGg9P/x5KN0FWPpx8Axx8dvhkX0kpqdJl5uabb+aHP/wh2dnbv7K/UaNGLF26dLeCSVKlfLUG/nMeLH8hHO9zPPS/BfJb73g7SbFX6TJz7rnn1kQOSdo92fmwqQDqNYATr4euFzgbI9URfpyBpPgq+AT2aAbp9SAjC37wD6iXDY3aRp1MUi3y4wwkxU8QwKv3wG2Hw3Pjtixv1tEiI9VBlhlJ8VK4Cu77EUy9BErWw4ezwwt/JdVZnmaSFA9BAG/8B574dXhtTHoWHH8V9BgKaelRp5MUIcuMpOT31Wfw2DB457Fw3OpQGDAhPK1Ug3IyM1h2Q78a/RmSdp9lRlLy2/QlvD8L0upBryug53BI99eXpJC/DSQlp9Li8A4lgCb7Qf9bw5mYFgdFm0tS0vECYEnVrqiklLYjptF2xDSKSkor/w3eeRz+cjAsf3nLsi4/tMhI2ibLjKTksfFLmDwYHjgb1q+C58dFnUhSDHiaSVJyeH8WPHIJrF8JJODIS+C4q6JOJSkGLDOSolW8HqZfBa/eHY4bt4cB42GvI6LNJSk2LDOSovXOtC1F5vBB0PsayMyJNpOkWLHMSIpWl7Ng+YvQeSC0OybqNJJiyAuAJdWu5S/D3aeFT/GF8JOtT/uLRUZSlVlmpCSy27c0J7PNm2DG7+AfJ8LSuTD7xqgTSUoRnmaSVPM+WRDecr12STg+5Bzo9ZtoM0lKGZYZSTWmHqXUmzMaXhgHQRns0Tw8pdTh5KijSUohlhlJNebyjP9S7/lHw0HnH8Apf4KcxtGGkpRyvGZGUo2ZWNqP8iYd4Yd3ww/utMhIqhGWGUnVZ8078PT1EAQArCOPZT+aCZ0GRJtLUkqzzEjafeVl8Pxf4G9Hw9w/8fBjj1Z8qc/Nz/LgvOURhpOU6iwzknbP2vfhHyfBzKuhrIRVbQdw9fysii+XB3Dlw4tZVbAxwpCSUpllRlLVlJfDSxNgQk/4+BXIzIX+t7K0502UB1uvWhYELFtbFE1OSSnPu5kkVc2UwfDGA+H79sdC/1uhYRvaFWwkLcFWhSY9kaBtEz9vSVLNcGZGUtUccjZk7gH9boJzp0DDNgC0zK/PqP6dKlZLS8DoMzvTMr9+REElpTrLjKRdU/AxvDtjy7j9sTBsEXT/Wfj5St8wsGvrivezLu/FWd33qqWQkuoiy4yUpFYXbIo6QigIYOG/4fYe8NAFsG7Zlq/twnNjWuRn11w2ScIyIyWVSQs+rnjfZ+yc6G9pXr8a7j8bHrkYiguhacfwNmxJSiKWGSlJrCrYyDVT36wYR3pLcxDAoofgtsPh3ScgPRN6XwMXToc996n9PJK0A97NJCWJpWs3bPeW5lq9eDYI4KEL4c2Hw3GLLnDG36D5gbWXQZIqwTIjJYl2TRokxy3NiQQ02hvSMuCYX8PRv4T0erWbQZIqwdNMUpKI9Jbmoi+2vrD32JHwi9lw7AiLjKSkZ5mRkkgktzS/Oz28U+m/F0BZabgsIwtaHFTzP1uSqoGnmaQkVeO3NG8qgOlXwsJ7w3FWLqxfVfHwO0mKC8uMVBd98DQ8cgkUfgwkoMcQOP63UM+n9EqKH8uMVJds3gjTr4L5d4bjRu1gwO2w95HR5pKk3RCra2bGjBlDIpFg2LBhUUeR4imtHqx6LXzf/ecw+HmLjKTYi83MzLx585g4cSJdunSJOooULyVFkJYeXtSbngEDJsD6leFnK0lSCojFzMxXX33FOeecwx133EGjRo2ijiPFx4p58Lej4ZnRW5Y13b/Gi0xOZgbLbujHshv6kZMZm/9mkhRTsSgzQ4YMoV+/fvTp02en6xYXF1NYWLjVS6pzSoth5jXwj77w+fvhRxOUbIg6lSTViKT/T6YHHniAV199lXnz5u3S+mPGjGHUqFE1nEpKYisXwuTB8Nnb4bjLWXDyjZDZINpcklRDknpmZsWKFVx22WXce++9ZGfv2jM3Ro4cSUFBQcVrxYoVNZxSShKlJeHppDt6h0WmQVM4699w5kSo7+lZSakrqWdmFixYwJo1a+jatWvFsrKyMubOncutt95KcXEx6enpW22TlZVFVlZWbUeVovfVp/DibRCUwYEDoN9YaLBn1KkkqcYldZnp3bs3ixYt2mrZBRdcQMeOHbniiiu+U2SkOicIwg+GhPDJvf3GhncsdR4YbS5JqkVJXWZyc3Pp3LnzVssaNGjAnnvu+Z3lUp3z2bvwyMVw3FWwz3HhsoPPijaTJEUgqcuMVNd8fUvzDpWXwUvj4enfQ+kmmPFbGPTclhkaSapjYldmZs+eHXUEKTqffwCPDIHlL4bjfXpD/1ssMpLqtNiVGalOKi8PP09p5tWwuQgy94C+10PX8y0ykuo8y4wUB0tnw+O/Ct+3PRpOvw0a7R1pJElKFpYZKQ7aHwcH/xhaHQrdfwZpSf2IKEmqVf5GlJJR4UqY9HPYsDYcJxJwxng4/BcWGUn6FmdmpGQSBPDGg/DEb2BTQbhs4B3RZpKkJGeZkZLFV2vg0WGwZFo4bvV9OPqXkUaSpDiwzEjJYPHDMO2XsPELSKsHx14BRw0Pn+YrSdohf1NKUZv/D3hsePi++UHhtTEtDoo2kyTFiFcSSlHrPBAatYVjfgM/f9oiI0mV5MyMVNs2roOF90KPoeFdStn5cPFLUK9+1MkkKZYsM1Jtem8WTB0K61dBVm74BF+wyEjSbrDMSLWheD1MvwpevTscN94Hmh0YbSZJShGWGammfTgHHhkKBcvD8eGDoffVkJkTbS5JShGWGakmPXczzLo2fN9wbxhwO7TtGWkkSUo1lhmlhKKSUg68ejoAb113IjmZSfJPe++jIJEOXc+DE34PWXtEnUiSUk6S/MaXUsTmjfDJgi2zL20Og0vmQ+P20eaSpBTmc2ak6vLxAvjbMfCvM+Gzd7cst8hIUo2yzEi7q7QYnvo93HkCrH0X6jeEDWuiTiVJdYanmaTdseoNmDIYPl0cjjv/AE75E+Q0jjaXJNUhlhmpqp4dC8/8AcpLIWdPOPVmOPD0qFNJUp1jmZGqqrwsLDIHnAb9boY9mkadSJLqJMuMtKvKy+CrNZDXMhz3HA4tOsP+J4WfsSRJioQXAEu7Yu378I8T4d4zwwt+AdIzoMPJFhlJiphlRtqR8nJ4aTxM6Akfz4OCj+HTN6NOJUn6Bk8zSduzbhlMGQIfPReO2x8H/W+Bhm0ijSVJ2pplRvq2IIAFd8H038LmDVCvAfT9PXS70FNKkpSELDPStwUBLJoUFpm9e8Lpt0LjdlGnkiRth2VGKWd1wSbaN63kBzoGQXibdXo9SEuDAbfBkifhsF+EY0lS0vK3tFLCpAUfV7zvM3YOD85bvusbr18N9/8/mH7llmWN2sIRgywykhQD/qZW7K0q2Mg1U7fcYVQewJUPL2ZVwcYdbxgE8MZ/4bbD4d0nYcHdULiyhtNKkqqbp5kUe0vXbqA82HpZWRCwbG0RLfPrb3ujrz6DacPh7UfDccuDYcAEyGtVs2ElSdXOMqPYa9ekAWkJtio06YkEbZvkbHuDt6bCY8OhaC2kZcAxv4GjLw+vl5EkxY6nmRR7LfPrM6p/p4pxWgJGn9l527MyG7+EqZeERaZZJ/j503DsFRYZSYoxZ2aUEgZ2bc3vHgmvm5l1ea/t381UvyH0uwnWvAW9roCMrNoLKUmqEZYZpZwW+dlbBpsKwruUOvSDjqeEyw76QTTBJEk1wjKj1PXB0/DIUCj8BN6bBfscD/Wyd76dJClWLDNKPSVfwYxRMP8f4bhROxgw3iIjSSnKMqOUcnjibbL/PhK+/Chc0P3ncMIoyGwQbTBJUo2xzChl7Jv4mPszryftywDy24SfqdT+2KhjSZJqmGVGKeP9oDWTyo7mjG57k3HSaMjOizqSJKkW+JwZxdfmTfD0H6BwVcWiK0p/Qckp4ywyklSHODOjeFq5ECYPgs/egVWvwQ/uA6Dcfi5JdY5lRvFSWgJz/wTP3gRBGTRoCt8/DxKJqJNJkiJimVF8rF4MUwbB6kXhuNMZcMpN0GBPKCmNNpskKTKWGcXD+0/BfWdB+Wao3zj8SILOZ0adSpKUBCwzioe9joCGbaBpRzh1HOQ23+rLOZkZLLuhXzTZJEmRsswoOZWXweKHofNASEsLH3p34Qxo0MTrYyRJW7HMKPl8/gFMuRhWvARFa+GIweHyPZpGm0uSlJQsM0oe5eUw/06YeTVsLoLMPSDL58VIknbMMqPksO4jmDoUls4Nx22PhtNvg0Z7R5tLkpT0LDOK3ltTYcrg8NOu6+VAn1HQ/WfhtTKSJO2EZUbRa9QWSjdBmyNgwO2w5z5RJ5IkxYhlRrUvCGDN29D8wHDcsgtc8AR8ryukpUebTZIUO87jq3at/xQeOAf+djSsemPL8jaHWWQkSVVimVHtWfww3H4ELJkGJGD1GzvdRJKknfE0k2rehs/h8V/Cm5PDcYuDYMAEaNE52lySpJRgmVHNWvIETL0ENnwGiXQ45ldw9K8gIzPqZJKkFGGZUc36YmlYZJoeAGeMh1aHRp1IkpRiLDOqfsXrISs3fH/4IEivB4eeC/Wyo80lSUpJXgCs6rOpMDyl9LdeUFIULktLg8N+bpGRJNUYy0wdV1RSStsR02g7YhpFJaVV/0YfzoHxR8Gr98AXH8IHT1VfSEmSdsDTTNo9JRtg5jUw745w3HBvGDAe2h4VbS5JUp1hmVHVffRi+JlK65aG427/BydcB1l7RJtLklSnWGZUdc+PC4tM3vfg9Fthn+OjTiRJqoMsM6qcIIBEInx/6jh49s/Q+2rIzo80liSp7krqC4DHjBlD9+7dyc3NpVmzZgwYMIAlS5ZEHatuKi2Gp66DRy/dsiyvJfS7ySIjSYpUUpeZOXPmMGTIEF566SVmzpxJaWkpffv2ZcOGDVFHq1tWvQ4Tj4NnbwrvVlq5MOpEkiRVSOrTTE8++eRW47vuuotmzZqxYMECjjnmmIhS1SFlm8MCM/dPUF4KOU3g1LE+xVeSlFSSusx8W0FBAQCNGzfe7jrFxcUUFxdXjAsLC2s8V0pa8zZMviiclQE4oD/0Gwt7NI02lyRJ35LUp5m+KQgCLr/8cnr27Ennztv/tOUxY8aQn59f8WrTpk0tpkwRZZvh3z8Mi0x2Qxh4J/zoHouMJCkpxabMDB06lDfeeIP7779/h+uNHDmSgoKCiteKFStqKWEKSa8HJ98I+58EQ16Gg36w5Q4mSZKSTCxOM11yySVMnTqVuXPn0rp16x2um5WVRVZWVi0lSw0JyrkgfTrpb22EQ34QLuzYDzqcYomRJCW9pC4zQRBwySWXMHnyZGbPnk27du2ijpRyEuuWcVu9v9CQDXz++LO02u9YaNDkf1+0yEiSkl9Sl5khQ4Zw33338cgjj5Cbm8vq1asByM/Pp379+hGni7kggPl38shjM7hy82WUk0ba5oAxbxZx1mFRh5MkadclgiAIog6xPYntzAzcddddnH/++bv0PQoLC8nPz6egoIC8vLxqTBdjX66AqUNZ9cEbHFX8V8q/celUeiLBcyOOo2W+ZVGSFJ3K/P1O6pmZJO5Z8bVhLYw/CooLWJo4ZKsiA1AWBCxbW2SZkSTFRmzuZlI1adAEuvwIWh9Gu/MmkPatya/0RIK2TXKiySZJUhVYZlJdEMAb/4V1y7Ys63s9XPgkLdsdwKj+nSoWpyVg9JmdnZWRJMWKZSaVffUZPPgTePhnMGUIlJeHy+tlQ1o6AAO7brnVfdblvTir+15RJJUkqcqS+poZ7Ya3HoHHhkPR55CWAe17QVDOjvpri/zs2ssnSVI1scykmqIv4PFfw+KHwnHzzjBgPLTsEm0uSZJqiGUmlax5G+45Hb76FBLp0HM49LoCMjKjTiZJUo2xzKSSxu0hZ8/wwyHPGA/f6xp1IkmSapxlJu4+ehFad4f0DMjIgrMfgD2ahxf5SpJUB3g3UxUVlZTSdsQ02o6YRlFJae0HKF4Pjw6Du06C52/esrzR3hYZSVKd4sxMHC19Fh65GL5cHo43fhlpHEmSomSZiZOSInhqFLw8IRzn7wWn3xredi1JUh1lmYmLla/BQxfCFx+E467nh0/yzcqNMpUkSZGzzMRFvRwo+BhyW0H/W2C/PlEnkiQpKVhmktn61ZDbInzfdH/4f/dB625Qv2GksSRJSibezZSMSkvg6eth3EGw4pUty/frY5GRJOlbnJlJNqsXweTB8OmicPzOY9DmsBr7cTmZGSy7oV+NfX9JkmqaZSZZlJWGz4uZfSOUb4b6jaHfTdD5zKiTSZKU1CwzyeCzJTB5EKx8NRx3PBVOvRn2aBZtLkmSYsAykwyWvxgWmex8OPlP0OVHkEhEnUqSpFiwzESlrDT8PCWA758X3rn0/Z9CXqtoc0mSFDPezVTbysvh5b/B+B6wqSBclkjAsSMsMpIkVYFlphqsLti0ayuu+wju6Q9P/AbWvgsL7q7ZYJIk1QGWmSqatODjivd9xs7hwXnLt79yEMD8u2D8kbDs2fBpvqf8GXoMrYWkkiSlNq+ZqYJVBRu5ZuqbFePyAK58eDHH7N+Ulvn1t1654BOYegl88FQ43qsHnH4b7LlPLSaWJCl1WWaqYOnaDZQHWy8rCwKWrS36bpl55g9hkUnPgt5XwxGDIS299sJKkpTiLDNV0K5JA9ISbFVo0hMJ2jbJ+e7KJ/weij4P/7fp/rUXUpKkOsJrZqqgZX59RvXvVDFOS8DoMzuHszKLJ8Gjl4XXyQA02BN+/KBFRpKkGmKZqaKBXVtXvJ91eS/OOrAB/Oc8eOhCWPBPePfJ6MJJklSHeJqpGnxv9VPw5C9hw2eQlgFH/wr27RN1LEmS6gTLzG7I4yuurXcPWZOeCxc0PQDOmACtDok0lyRJdYllpqqCgH9m/pHvp71PkEgjcdRlcOxIyMiKOpkkSXWK18xUVSLBTaU/5P3yVhT/9HHoc61FRpKkCFhmdsPz5QdxYsmNlH+ve9RRJEmqsywzu6kMH4AnSVKULDOSJCnWLDOSJCnWLDOSJCnWLDOSJCnWLDOSJCnWfGheFeVkZrDshn5Rx5Akqc5zZkaSJMWaZUaSJMWaZUaSJMWaZUaSJMWaZUaSJMWaZUaSJMWaZUaSJMWaZUaSJMWaZUaSJMWaZUaSJMWaZUaSJMWaZUaSJMWaZUaSJMWaZUaSJMWaZUaSJMVaRtQBaloQBAAUFhZGnESSJO2qr/9uf/13fEdSvsysX78egDZt2kScRJIkVdb69evJz8/f4TqJYFcqT4yVl5ezcuVKcnNzSSQS1fq9CwsLadOmDStWrCAvL69av3cycP/iL9X30f2Lv1TfR/ev6oIgYP369bRq1Yq0tB1fFZPyMzNpaWm0bt26Rn9GXl5eSv4j/Zr7F3+pvo/uX/yl+j66f1WzsxmZr3kBsCRJijXLjCRJijXLzG7IysrimmuuISsrK+ooNcL9i79U30f3L/5SfR/dv9qR8hcAS5Kk1ObMjCRJijXLjCRJijXLjCRJijXLjCRJijXLzA7cfvvttGvXjuzsbLp27cqzzz67w/XnzJlD165dyc7Opn379kyYMKGWklZdZfZx9uzZJBKJ77zeeeedWky86+bOnctpp51Gq1atSCQSTJkyZafbxOkYVnb/4nb8xowZQ/fu3cnNzaVZs2YMGDCAJUuW7HS7uBzDquxf3I7h+PHj6dKlS8UD1Xr06METTzyxw23icvyg8vsXt+P3bWPGjCGRSDBs2LAdrhfFMbTMbMeDDz7IsGHDuOqqq1i4cCFHH300J598MsuXL9/m+kuXLuWUU07h6KOPZuHChVx55ZVceumlTJo0qZaT77rK7uPXlixZwqpVqype++23Xy0lrpwNGzZw8MEHc+utt+7S+nE7hpXdv6/F5fjNmTOHIUOG8NJLLzFz5kxKS0vp27cvGzZs2O42cTqGVdm/r8XlGLZu3ZobbriB+fPnM3/+fI4//nhOP/103nzzzW2uH6fjB5Xfv6/F5fh907x585g4cSJdunTZ4XqRHcNA23TYYYcFgwYN2mpZx44dgxEjRmxz/d/85jdBx44dt1p20UUXBUcccUSNZdxdld3HZ555JgCCdevW1UK66gUEkydP3uE6cTyGX9uV/Yvz8QuCIFizZk0ABHPmzNnuOnE+hruyf3E/hkEQBI0aNQr+/ve/b/NrcT5+X9vR/sX1+K1fvz7Yb7/9gpkzZwa9evUKLrvssu2uG9UxdGZmG0pKSliwYAF9+/bdannfvn154YUXtrnNiy+++J31TzzxRObPn8/mzZtrLGtVVWUfv3booYfSsmVLevfuzTPPPFOTMWtV3I5hVcX1+BUUFADQuHHj7a4T52O4K/v3tTgew7KyMh544AE2bNhAjx49trlOnI/fruzf1+J2/IYMGUK/fv3o06fPTteN6hhaZrZh7dq1lJWV0bx5862WN2/enNWrV29zm9WrV29z/dLSUtauXVtjWauqKvvYsmVLJk6cyKRJk3j44Yfp0KEDvXv3Zu7cubURucbF7RhWVpyPXxAEXH755fTs2ZPOnTtvd724HsNd3b84HsNFixaxxx57kJWVxaBBg5g8eTIHHnjgNteN4/GrzP7F8fg98MADvPrqq4wZM2aX1o/qGKb8p2bvjkQisdU4CILvLNvZ+ttankwqs48dOnSgQ4cOFeMePXqwYsUK/vznP3PMMcfUaM7aEsdjuKvifPyGDh3KG2+8wXPPPbfTdeN4DHd1/+J4DDt06MBrr73Gl19+yaRJkzjvvPOYM2fOdv/gx+34VWb/4nb8VqxYwWWXXcaMGTPIzs7e5e2iOIbOzGxDkyZNSE9P/84MxZo1a77TOL/WokWLba6fkZHBnnvuWWNZq6oq+7gtRxxxBO+99151x4tE3I5hdYjD8bvkkkuYOnUqzzzzDK1bt97hunE8hpXZv21J9mOYmZnJvvvuS7du3RgzZgwHH3wwf/nLX7a5bhyPX2X2b1uS+fgtWLCANWvW0LVrVzIyMsjIyGDOnDn89a9/JSMjg7Kysu9sE9UxtMxsQ2ZmJl27dmXmzJlbLZ85cyZHHnnkNrfp0aPHd9afMWMG3bp1o169ejWWtaqqso/bsnDhQlq2bFnd8SIRt2NYHZL5+AVBwNChQ3n44Yd5+umnadeu3U63idMxrMr+bUsyH8NtCYKA4uLibX4tTsdve3a0f9uSzMevd+/eLFq0iNdee63i1a1bN8455xxee+010tPTv7NNZMewRi8vjrEHHnggqFevXnDnnXcGb731VjBs2LCgQYMGwbJly4IgCIIRI0YE5557bsX6H374YZCTkxMMHz48eOutt4I777wzqFevXvDQQw9FtQs7Vdl9vPnmm4PJkycH7777brB48eJgxIgRARBMmjQpql3YofXr1wcLFy4MFi5cGADB2LFjg4ULFwYfffRREATxP4aV3b+4Hb/BgwcH+fn5wezZs4NVq1ZVvIqKiirWifMxrMr+xe0Yjhw5Mpg7d26wdOnS4I033giuvPLKIC0tLZgxY0YQBPE+fkFQ+f2L2/Hblm/fzZQsx9AyswO33XZbsPfeeweZmZnB97///a1umTzvvPOCXr16bbX+7Nmzg0MPPTTIzMwM2rZtG4wfP76WE1deZfbxxhtvDPbZZ58gOzs7aNSoUdCzZ89g2rRpEaTeNV/fBvnt13nnnRcEQfyPYWX3L27Hb1v7BgR33XVXxTpxPoZV2b+4HcMLL7yw4vdL06ZNg969e1f8oQ+CeB+/IKj8/sXt+G3Lt8tMshzDRBD878ocSZKkGPKaGUmSFGuWGUmSFGuWGUmSFGuWGUmSFGuWGUmSFGuWGUmSFGuWGUmSFGuWGUmSFGuWGUmSFGuWGUmSFGuWGUmx8tlnn9GiRQtGjx5dsezll18mMzOTGTNmRJhMUlT8bCZJsfP4448zYMAAXnjhBTp27Mihhx5Kv379GDduXNTRJEXAMiMploYMGcKsWbPo3r07r7/+OvPmzSM7OzvqWJIiYJmRFEsbN26kc+fOrFixgvnz59OlS5eoI0mKiNfMSIqlDz/8kJUrV1JeXs5HH30UdRxJEXJmRlLslJSUcNhhh3HIIYfQsWNHxo4dy6JFi2jevHnU0SRFwDIjKXZ+/etf89BDD/H666+zxx57cNxxx5Gbm8tjjz0WdTRJEfA0k6RYmT17NuPGjeNf//oXeXl5pKWl8a9//YvnnnuO8ePHRx1PUgScmZEkSbHmzIwkSYo1y4wkSYo1y4wkSYo1y4wkSYo1y4wkSYo1y4wkSYo1y4wkSYo1y4wkSYo1y4wkSYo1y4wkSYo1y4wkSYo1y4wkSYq1/w+B+I5fUZ/qLQAAAABJRU5ErkJggg==",
- "text/plain": [
- "<Figure size 640x480 with 1 Axes>"
- ]
- },
- "metadata": {},
- "output_type": "display_data"
- }
- ],
- "source": [
- "#hideme\n",
- "import numpy as np\n",
- "import scipy.stats as stats\n",
- "import matplotlib.pyplot as plt \n",
- "\n",
- "def f(x):\n",
- " return 2*x + 1\n",
- "\n",
- "n = 10\n",
- "xs = np.linspace(0,4,n)\n",
- "sigma_y=0.4\n",
- "ys = stats.multivariate_normal.rvs(f(xs), np.eye(n)*sigma_y**2, 1, random_state=42)\n",
- " \n",
- "x_axis = np.linspace(0,4,100)\n",
- "plt.errorbar(xs,ys,yerr=sigma_y,fmt=\".\")\n",
- "plt.plot(x_axis, f(x_axis),'--')\n",
- "plt.xlabel(\"x\")\n",
- "plt.ylabel(\"y\")\n",
- "plt.savefig(\"line.png\")\n",
- "plt.show()"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "64e67452-e6bd-442c-a174-e12bdb18dba0",
- "metadata": {
- "slideshow": {
- "slide_type": ""
- },
- "tags": []
- },
- "source": [
- "### Methode der kleinsten Quadrate \n",
- "\n",
- "$$\\chi^2 = \\sum_i \\left(\\frac{y_i - \\hat y(x)}{\\sigma_i}\\right)^2$$\n",
- "quantifiziert die Übereinstimmung von Modell zu Daten \n",
- "$\\rightarrow$ $\\hat m$ und $\\hat a$ sollten $\\chi^2$ minimieren.\n",
- "<img src=\"./figures/11/line.png\" style=\"width:80.0%\"\n",
- "alt=\"image\" />"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "bdc0949d-791e-4e29-9adc-169578e62821",
- "metadata": {
- "slideshow": {
- "slide_type": "slide"
- },
- "tags": []
- },
- "source": [
- "### Methode der kleinsten Quadrate II \n",
- "\n",
- "Minimiere\n",
- "$\\chi^2 = \\sum_i \\left(\\frac{y_i - \\hat y(x)}{\\sigma_i}\\right)^2 = \\sum_i \\frac{(y_i - m x_i - a)^2}{\\sigma_i^2}$:\n",
- "\n",
- "Erste Ableitung ist Null: \n",
- "\n",
- "$$\\begin{aligned}\n",
- " \\frac{d\\chi^2}{dm} &=& -2\\sum_i x_i\\frac {y_i -\\hat m x_i - \\hat a}{\\sigma_i^2} = 0\\\\\n",
- " \\frac{d\\chi^2}{da} &=& -2\\sum_i \\frac{y_i - \\hat m x_i - \\hat a}{\\sigma_i^2} = 0 \\\\\n",
- " \\sum_i\\frac{x_iy_i}{\\sigma_i^2} - \\hat m \\sum_i\\frac{x_i^2}{\\sigma_i^2}- \\hat a \\sum_i \\frac{x_i}{\\sigma_i^2} &=& 0 \\\\\n",
- " \\sum_i\\frac{y_i}{\\sigma_i^2} - \\hat m \\sum_i\\frac{x_i}{\\sigma_i^2}- \\hat a \\sum_i \\frac{1}{\\sigma_i^2} &=& 0 \n",
- "\\end{aligned}$$"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "322c67fc-ddd1-4830-a5f5-6f118ef54c4c",
- "metadata": {
- "slideshow": {
- "slide_type": "slide"
- },
- "tags": []
- },
- "source": [
- "### Methode der kleinsten Quadrate III \n",
- "\n",
- "Minimiere\n",
- "$\\chi^2 = \\sum_i \\left(\\frac{y_i - \\hat y(x)}{\\sigma_i}\\right)^2 = \\sum_i \\frac{(y_i - m x_i - a)^2}{\\sigma_i^2}$: \n",
- "$$\\begin{aligned}\n",
- " \\sum_i\\frac{x_iy_i}{\\sigma_i^2} - \\hat m \\sum_i\\frac{x_i^2}{\\sigma_i^2}- \\hat a \\sum_i \\frac{x_i}{\\sigma_i^2} &=& 0 \\\\\n",
- " \\sum_i\\frac{y_i}{\\sigma_i^2} - \\hat m \\sum_i\\frac{x_i}{\\sigma_i^2}- \\hat a \\sum_i \\frac{1}{\\sigma_i^2} &=& 0 \n",
- "\\end{aligned}$$ mit\n",
- "$\\frac{1}{\\sum_i 1/\\sigma_i^2} \\sum_i \\frac{f}{\\sigma_i^2} = \\langle f \\rangle$: \n",
- "$$\\begin{aligned}\n",
- " \\langle xy \\rangle -\\langle x^2 \\rangle \\hat m& - \\langle x \\rangle \\hat a&= 0\\\\\n",
- " \\langle y \\rangle - \\langle x \\rangle \\hat m& - \\hat a& = 0 \n",
- "\\end{aligned}$$"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "fb25687c-4410-4281-b540-39369732fb26",
- "metadata": {
- "slideshow": {
- "slide_type": "slide"
- },
- "tags": []
- },
- "source": [
- "### Methode der kleinsten Quadrate IV \n",
- "\n",
- "$$\\begin{aligned}\n",
- " \\hat m&=&\\frac{\\langle xy \\rangle - \\langle y \\rangle\\langle x \\rangle}{\\langle x^2 \\rangle - \\langle x \\rangle^2} = \\frac{1}{\\sum_i 1/\\sigma_i^2} \\sum_i \\frac{x_i - \\langle x \\rangle}{\\sigma_i^2(\\langle x^2 \\rangle - \\langle x \\rangle^2)}y_i\\\\\n",
- " \\hat a &=& \\frac{ \\langle y \\rangle \\langle x^2 \\rangle- \\langle y \\rangle \\langle x \\rangle^2- \\langle x \\rangle \\langle xy \\rangle+ \\langle y \\rangle \\langle x \\rangle^2}{ \\langle x^2 \\rangle- \\langle x \\rangle^2}\\\\\n",
- " &=& \\frac{ \\langle y \\rangle \\langle x^2 \\rangle - \\langle x \\rangle \\langle xy \\rangle}{ \\langle x^2 \\rangle - \\langle x \\rangle^2} = \\frac{1}{\\sum_i 1/\\sigma_i^2} \\sum_i \\frac{\\langle x^2 \\rangle - \\langle x \\rangle x_i}{\\sigma_i^2(\\langle x^2 \\rangle - \\langle x \\rangle^2)}y_i\n",
- "\\end{aligned}$$"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "b08ade05-e5eb-4a7e-9315-31a45c51aadb",
- "metadata": {
- "slideshow": {
- "slide_type": ""
- },
- "tags": []
- },
- "source": [
- "## Fehler\n",
- "\n",
- "### Fehler \n",
- "\n",
- "$$\\begin{aligned}\n",
- "V(\\hat m) = \\sum_i \\left(\\frac{d\\hat m}{y_i}\\sigma_i\\right)^2\\text{; }\\frac{d\\hat m}{y_i} & = & \\frac{1}{\\sum_i 1/\\sigma_i^2} \\frac{x_i - \\langle x \\rangle}{\\sigma_i^2(\\langle x^2 \\rangle - \\langle x \\rangle^2)} \\\\\n",
- "V(\\hat a) = \\sum_i \\left(\\frac{d\\hat a}{y_i}\\sigma_i\\right)^2\\text{; }\\frac{d\\hat a}{y_i} & = & \\frac{1}{\\sum_i 1/\\sigma_i^2} \\frac{\\langle x^2 \\rangle - \\langle x \\rangle x_i}{\\sigma_i^2(\\langle x^2 \\rangle - \\langle x \\rangle^2)}\n",
- "\\end{aligned}$$ $$\\begin{aligned}\n",
- "V(\\hat m) &=& \\left(\\frac{1}{\\sum_i 1/\\sigma_i^2}\\right)^2 \\sum_i \\left(\\frac{x_i - \\langle x \\rangle}{\\sigma_i^2(\\langle x^2 \\rangle - \\langle x \\rangle^2)}\\right)^2 \\sigma_i^2 \\\\\n",
- "&=& \\frac{1}{\\sum_i 1/\\sigma_i^2} \\frac{\\langle x^2 \\rangle - 2\\langle x \\rangle \\langle x \\rangle + \\langle x \\rangle^2}{(\\langle x^2 \\rangle - \\langle x \\rangle^2)^2} \n",
- "= \\frac{1}{\\sum_i 1/\\sigma_i^2} \\frac{1}{\\langle x^2 \\rangle - \\langle x \\rangle^2} \\\\\n",
- "V(\\hat a) &=& \\frac{1}{\\sum_i 1/\\sigma_i^2} \\frac{\\langle x^2 \\rangle^2 - 2\\langle x^2 \\rangle\\langle x \\rangle^2 + \\langle x^2 \\rangle\\langle x \\rangle^2}{(\\langle x^2 \\rangle - \\langle x \\rangle^2)^2}\n",
- "= \\frac{1}{\\sum_i 1/\\sigma_i^2} \\frac{\\langle x^2 \\rangle}{\\langle x^2 \\rangle - \\langle x \\rangle^2}\n",
- "\\end{aligned}$$"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "d8828d04-0af8-4dc3-a846-24acbc9a0f8e",
- "metadata": {
- "slideshow": {
- "slide_type": ""
- },
- "tags": []
- },
- "source": [
- "### Korrelation \n",
- "\n",
- "$$\\begin{aligned}\n",
- "V(\\hat m) &=& \\frac{1}{\\sum_i 1/\\sigma_i^2} \\frac{1}{\\langle x^2 \\rangle - \\langle x \\rangle^2} \\\\\n",
- "V(\\hat a) &=& \\frac{1}{\\sum_i 1/\\sigma_i^2} \\frac{\\langle x^2 \\rangle}{\\langle x^2 \\rangle - \\langle x \\rangle^2}\\\\\n",
- "\\text{cov}(\\hat m, \\hat a) &=&= \\frac{1}{\\sum_i 1/\\sigma_i^2} \\frac{\\langle (x-\\langle x \\rangle)(\\langle x^2 \\rangle - \\langle x \\rangle x)\\rangle}{(\\langle x^2 \\rangle - \\langle x \\rangle^2)^2}\\\\\n",
- "&=& \\frac{1}{\\sum_i 1/\\sigma_i^2} \\frac{\\langle x^2 \\rangle \\langle x \\rangle - \\langle x \\rangle \\langle x^2 \\rangle - \\langle x \\rangle \\langle x^2 \\rangle + \\langle x \\rangle^2\\langle x \\rangle}{(\\langle x^2 \\rangle - \\langle x \\rangle^2)^2}\\\\\n",
- "&=& - \\frac{1}{\\sum_i 1/\\sigma_i^2} \\frac{\\langle x \\rangle}{\\langle x^2 \\rangle - \\langle x \\rangle^2}\n",
- "\\end{aligned}$$\n",
- "\n",
- "### Beispiel in Jupyter "
- ]
- },
- {
- "cell_type": "markdown",
- "id": "424bdd1f-53bf-422b-bc4a-0702231b976d",
- "metadata": {
- "slideshow": {
- "slide_type": ""
- },
- "tags": []
- },
- "source": [
- "### Minimales $\\chi^2$ \n",
- "\n",
- "$$\\begin{aligned}\n",
- " \\chi^2 &=& \\sum_i \\frac{(y_i - \\hat m x_i - \\hat a)^2}{\\sigma_i^2} = \\sum_i \\frac{\\left[y_i - \\frac{\\langle xy \\rangle - \\langle y \\rangle\\langle x \\rangle}{\\langle x^2 \\rangle - \\langle x \\rangle^2} x_i - \\frac{ \\langle y \\rangle \\langle x^2 \\rangle - \\langle x \\rangle \\langle xy \\rangle}{ \\langle x^2 \\rangle - \\langle x \\rangle^2} \\right]^2}{\\sigma_i^2}\\\\\n",
- " & = & \\sum_i \\frac{\\left[(\\langle x^2 \\rangle - \\langle x \\rangle^2)y_i - (\\langle xy \\rangle - \\langle y \\rangle\\langle x \\rangle)x_i - \\langle y \\rangle \\langle x^2 \\rangle + \\langle x \\rangle \\langle xy \\rangle\\right]^2}{\\sigma_i^2 ( \\langle x^2 \\rangle - \\langle x \\rangle^2)^2} \\\\\n",
- " &=& \\dots\\\\\n",
- "& =& (\\sum_i \\frac{1}{\\sigma_i^2}) V(y) ( 1- \\rho^2_{xy})\n",
- "\\end{aligned}$$"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "e89f415e-8836-4b97-893c-a7335c3a21e5",
- "metadata": {
- "slideshow": {
- "slide_type": ""
- },
- "tags": []
- },
- "source": [
- "### Beispiel in Jupyter \n",
- "\n",
- "## In Python"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "7311c0ff-0ce0-4427-a50e-b6698100e454",
- "metadata": {
- "slideshow": {
- "slide_type": ""
- },
- "tags": []
- },
- "source": [
- "### Mit Python I\n",
- "\n",
- "Mit scipy.optimize:\n",
- "\n",
- "```\n",
- "import scipy.optimize as opti\n",
- "def fitf(x, m , a):\n",
- " return m*x + a\n",
- "pfit, Vfit = opti.curve_fit(fitf , xs, ys, \n",
- " sigma=[sigma_y]*len(ys),absolue_sigma=True)\n",
- "print(pfit, Vfit)\n",
- "```\n",
- "\n",
- "Vorsicht! Falsche Unsicherheit ohne `absolute_sigma=True`"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "a5fec52e-bd3a-4437-bb43-620de44939b2",
- "metadata": {
- "slideshow": {
- "slide_type": ""
- },
- "tags": []
- },
- "source": [
- "### Mit Python II\n",
- "\n",
- "Mit scipy.optimize:\n",
- "\n",
- "```\n",
- "def chi2(x, y, sy, a, m):\n",
- " my = m * x + a\n",
- " r = (y - my)/sy\n",
- " return np.sum(r**2)\n",
- " \n",
- "res = opti.minimize( lambda p: chi2(xs, ys, sigma_y, p[1], p[0]),x0=np.zeros(2))\n",
- "print(res.x, res.hess_inv*2)\n",
- "```"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "84b00301-c0d3-4595-858e-4f784d69c876",
- "metadata": {
- "slideshow": {
- "slide_type": "slide"
- },
- "tags": []
- },
- "source": [
- "### Inverse Hesse-Matrix und $\\chi^2$ \n",
- "\n",
- "$\\Delta \\chi2$ und Kovarianz Ellipse um Minimum gemäß Kovarianzmatrix\n",
- "genau bei $\\Delta \\chi^2 = 1$. \n",
- "$$1 = \\delta \\chi^2 = (\\vec a -\\hat \\vec a)^T V^{-1} (\\vec a-\\hat \\vec a)$$\n",
- "Mit\n",
- "$\\chi^2(\\vec a) = \\chi^2(\\hat \\vec a) + (\\vec a -\\hat \\vec a)^T V^{-1} (\\vec a-\\hat \\vec a)$\n",
- "und\n",
- "$H_{ij} = \\frac{\\partial^2 \\chi^2(\\vec a)}{\\partial a_i \\partial a_j}$ \n",
- "$$H_{ij} = \\frac{\\partial^2 (a_k -\\hat a_k) V^{-1}_{kl} (a_l -\\hat a_l)}{\\partial a_i \\partial a_j} = \\frac{\\partial( \\delta_{ik}V^{-1}_{kl} (a_l -\\hat a_l) + (a_k -\\hat a_k) V^{-1}_{kl} \\delta_{il})}{\\partial a_j}$$\n",
- "$$H_{ij} = \\delta_{ik}V^{-1}_{kl}\\delta_{lj} + \\delta_{jk}V^{-1}_{kl}\\delta_{il} = 2V^{-1}_{ij} \\text{ und } V_{ij} = 2 * H^{-1}_{ij}$$\n",
- "\n",
- "Vorsicht! Manche Algorithmen in `minimize` berechnen keine inverse\n",
- "Hesse-Matrix."
- ]
- },
- {
- "cell_type": "markdown",
- "id": "632583d4-1e97-498b-b8f3-91e259baa24d",
- "metadata": {
- "slideshow": {
- "slide_type": "slide"
- },
- "tags": []
- },
- "source": [
- "# Maximum-Likelihood \n",
- "\n",
- "Maximum-Likelihood (ML) Daten: $x_1,...,x_N$ \n",
- "Wahrscheinlichkeit der Daten für Modell mit Parametern $a$:\n",
- "$$P(x_1,...,x_N; a) = \\prod_i P(x_i ; a)$$\n",
- "\n",
- "Likelihoodfunktion: $$L(a) = \\prod_i P(x_i ; a)$$\n",
- "\n",
- "ML-Schätzer $\\hat a$: Maximum von $L(a)$:\n",
- "$$\\left.\\frac{dL}{da}\\right|_{a = \\hat a} = 0$$ (praktischer:\n",
- "Log-Likelihood: $-\\ln L = \\sum_i -\\ln P(x_i; a)$)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "dd0afbf7-8504-46f8-b3da-4fb33487a635",
- "metadata": {
- "slideshow": {
- "slide_type": "slide"
- },
- "tags": []
- },
- "source": [
- "### Beispiel\n",
- "\n",
- "\n",
- "$y(x) = mx + a$: Finde $\\hat m$ und $\\hat a$ Daten: $y_1,...,y_N$ und\n",
- "Modell: $$P(y_i; m, a) = G(y_i; \\mu = m x_i + a, \\sigma=\\sigma_i)$$\n",
- "$$L(m, a) = \\prod_i G(y_i; \\mu = m x_i + a, \\sigma=\\sigma_i)$$\n",
- "\n",
- "0.5 <img src=\"./figures/11/line.png\" alt=\"image\" />\n",
- "\n",
- "<img src=\"./figures/11/like_a.png\" style=\"width:49.0%\"\n",
- "alt=\"image\" />\n",
- "<img src=\"./figures/11/loglike_a.png\" style=\"width:49.0%\"\n",
- "alt=\"image\" />\n",
- "\n",
- "ML-Schätzer für Poisson $\\mu$ $$\\begin{aligned}\n",
- " L(\\mu) & = & \\prod_i^N P(k_i; \\mu) = \\prod_i^N \\frac{\\mu^{k_i}e^{-\\mu}}{k_i!}\\\\\n",
- " \\ln L(\\mu) & = & \\sum_{i=1}^N \\left( \\ln \\mu^{k_i} + \\ln e^{-\\mu} - \\ln k_i!\\right)\\\\\n",
- " & = & \\sum_{i=1}^N \\left( k_i \\ln \\mu -\\mu - \\ln k_i!\\right)\\\\\n",
- " 0 \\stackrel{!}{=} \\frac{d \\ln L(\\mu)}{d\\mu} \\Big|_{\\hat \\mu}& = & \\sum_{i=1}^N \\left( \\frac{k_i}{\\hat \\mu} - 1\\right) = \\sum_{i=1}^N \\frac{k_i}{\\hat\\mu} - N\\\\\n",
- " N & = & \\frac{1}{\\hat\\mu} \\sum_{i=1}^N k_i \\rightarrow \\hat\\mu = \\frac{1} {N} \\sum_{i=1}^N k_i\n",
- " \n",
- "\\end{aligned}$$"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "d9571970-772e-4e95-8474-82e0afb1dd68",
- "metadata": {
- "slideshow": {
- "slide_type": ""
- },
- "tags": []
- },
- "source": [
- "Varianz des ML-Schätzers\n",
- "\n",
- "Rao-Cramér-Frechet-Ungleichung: Schätzer $\\hat a$ mit Bias (Verzerrung)\n",
- "$b$\n",
- "$$V(\\hat a) \\geq \\frac{\\left(1+ \\frac{\\partial b}{\\partial a} \\right)^2}{E\\left[-\\frac{\\partial^2 \\ln L}{\\partial a^2}\\right]}$$\n",
- "Fisher-Information:\n",
- "$$I(\\hat a) = E\\left [-\\frac{\\partial^2 \\ln L}{\\partial a^2}\\right]$$\n",
- "\n",
- "ML-Schätzer für Poisson $V(\\hat \\mu)$ $$\\begin{aligned}\n",
- "V(\\hat \\mu) & \\geq &\\frac{\\left(1+ \\frac{\\partial b}{\\partial \\mu} \\right)^2}{E\\left[-\\frac{\\partial^2 \\ln L}{\\mu^2}\\right]} \\\\\n",
- " & = & \\frac{1}{E\\left[-\\frac{\\partial(\\sum_{i=1}^N \\frac{k_i}{\\mu} - N)}{\\partial \\mu^2}\\right]} \\\\\n",
- " & = & \\frac{1}{E\\left[-\\sum_{i=1}^N \\frac{-k_i}{\\hat \\mu^2}\\right]} = \\frac{1}{E\\left[\\sum_{i=1}^N \\frac{k_i}{\\hat \\mu^2}\\right]} \\\\\n",
- " & = & \\frac{1}{\\frac{1}{\\hat \\mu^2}E\\left[\\sum_{i=1}^N k_i \\right]} = \\frac{1}{\\frac{1}{\\hat \\mu^2}E\\left[N \\hat \\mu \\right]}\\\\\n",
- " & = & \\frac{\\hat \\mu}{N}\n",
- "\\end{aligned}$$\n",
- "\n",
- "Varianz für mehrere Parameter $\\vec \\theta$\n",
- "\n",
- "Für effizienten und erwartungstreuen Schätzer:\n",
- "$$\\left(V^{-1}\\right)_{ij} = E\\left[ -\\frac{\\partial^2 \\ln L(\\theta)}{\\partial \\theta_i \\partial \\theta_j}\\right]$$\n",
- "\n",
- "Näherung für große Datensätze:\n",
- "$$\\left(\\hat V^{-1}\\right)_{ij} = -\\frac{\\partial^2 \\ln L(\\theta)}{\\partial \\theta_i \\partial \\theta_j}\\Big|_{\\theta=\\hat \\theta} =$$\n",
- "\n",
- "Graphisch:\n",
- "$$\\ln L(\\theta) \\approx \\ln L(\\hat \\theta) + \\frac{\\partial \\ln L}{\\partial \\theta}\\Big|_{\\hat \\theta}(\\theta - \\hat \\theta) + \\frac{1}{2} \\frac{\\partial^2 \\ln L}{\\partial \\theta^2}(\\theta - \\hat \\theta)^2$$\n",
- "$$\\ln L(\\hat \\theta + \\sigma_\\theta) \\approx \\ln L(\\hat \\theta) + \\frac{1}{2} \\frac{\\partial^2 \\ln L}{\\partial \\theta^2}(\\sigma_\\theta)^2 = \\ln L(\\hat \\theta) - \\frac{1}{2}$$\n",
- "\n",
- "Zusammenhang ML und $\\chi^2$\n",
- "\n",
- "Likelihood-Quotient:\n",
- "$$\\lambda = -2 \\ln \\frac{L(\\hat \\theta)}{L(\\hat \\theta^\\prime_\\text{saturiert})}$$\n",
- "\n",
- "Mit Normalverteilung: $$\\begin{aligned}\n",
- "\\lambda &=& -2 \\ln \\frac{L(\\hat \\theta)}{L(\\hat \\theta^\\prime_\\text{saturiert})} = -2 \\ln \\frac{\\prod_i G(x_i; \\hat \\mu, \\sigma_i)}{\\prod_i G(x_i; x_i, \\sigma_i)}\\\\\n",
- "& = & -2 \\ln \\frac{\\frac{1}{\\sqrt{2\\pi}\\sigma_i}exp\\left(\\frac{(x_i-\\hat \\mu)^2}{2\\sigma_i^2}\\right)}{\\frac{1}{\\sqrt{2\\pi}\\sigma_i}exp\\left(\\frac{(x_i-x_i)^2}{2\\sigma_i^2}\\right)} = -2\\ln exp\\left(\\frac{(x_i-\\hat \\mu)^2}{2\\sigma_i^2}\\right) \\\\\n",
- "& = & -2 \\frac{(x_i-\\hat \\mu)^2}{2\\sigma_i^2} = \\chi^2 \\text{; also } \\ln L(\\theta) = - \\chi^2(\\theta) / 2 \n",
- "\\end{aligned}$$"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "7ae85d99-d6fa-409b-abb2-144cc24570db",
- "metadata": {
- "slideshow": {
- "slide_type": "slide"
- },
- "tags": []
- },
- "source": [
- "# Zusammenfassung und Ausblick\n",
- "\n",
- "## Zusammenfassung und Ausblick\n",
- "\n",
- "Zusammenfassung\n",
- "\n",
- "- Methode der kleinsten Quadrate ($\\chi^2$)\n",
- "\n",
- "- Maximum-Likelihood\n",
- "\n",
- "- Zusammenhang $\\chi^2$-ML\n",
- "\n",
- "- Minimierung\n",
- "\n",
- "- Literatur: \n",
- "\n",
- " - Glen Cowan, Statistical Data Analysis,\n",
- " [pdf](https://www.sherrytowers.com/cowan_statistical_data_analysis.pdf)\n",
- "\n",
- " - Roger John Barlow, Statistics: A Guide to the Use of Statistical\n",
- " Methods in the Physical Sciences,\n",
- " [Skript](https://arxiv.org/pdf/1905.12362.pdf)\n",
- "\n",
- " - Volker Blobel, Erich Lohrmann, Statistische und numerische\n",
- " Methoden der Datenanalyse,\n",
- " [pdf](https://www.desy.de/~sschmitt/blobel/eBuch.pdf)\n",
- "\n",
- "# Bibliography\n",
- "\n",
- "Bibliography"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3 (ipykernel)",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.9.20"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}