{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "92c14bbe-6493-4ca3-afa4-873bc2c980b9",
   "metadata": {},
   "source": [
    "# Chapter 14.5 - Variability of the Sample Mean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9538feb6-a0fb-4e3f-9a4f-5d45072870b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "from datascience import *\n",
    "import numpy as np\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plots"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4848b1e0-2e13-4068-b653-618074856d0d",
   "metadata": {},
   "outputs": [],
   "source": [
    "song_lengths = Table.read_table(\"top_spotify_songs_usa.csv\").select(\"duration_ms\")\n",
    "song_lengths = song_lengths.with_column(\"seconds\", song_lengths.column(\"duration_ms\") / 1000)\n",
    "song_lengths = song_lengths.drop(\"duration_ms\")\n",
    "song_lengths.show(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d0223a98-7216-4ef0-9b84-bc5554f9ef84",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Mean seconds:\", np.mean(song_lengths.column(\"seconds\")))\n",
    "print(\"Minimum seconds:\", np.min(song_lengths.column(\"seconds\")))\n",
    "print(\"Maximum seconds:\", np.max(song_lengths.column(\"seconds\")))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3402a4b1-ba27-4fc8-ba7c-016db0236233",
   "metadata": {},
   "outputs": [],
   "source": [
    "Table().with_column(\"Seconds\", song_lengths.column(\"seconds\")).hist(bins=np.arange(30, 521, 20))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "65bfa307-920b-4eba-8091-af58b3eb1e31",
   "metadata": {},
   "outputs": [],
   "source": [
    "def simulate_sample_mean(table, label, sample_size, repetitions):\n",
    "    means = make_array()\n",
    "    for _ in range(repetitions):\n",
    "        new_sample = table.sample(sample_size)\n",
    "        new_sample_mean = np.mean(new_sample.column(label))\n",
    "        means = np.append(means, new_sample_mean)\n",
    "    sample_means = Table().with_column('Sample Means', means)\n",
    "    \n",
    "    # Display empirical histogram and print all relevant quantities\n",
    "    sample_means.hist(bins=20)\n",
    "    plots.xlabel('Sample Means')\n",
    "    plots.title('Sample Size ' + str(sample_size))\n",
    "    population_std = np.std(table.column(label))\n",
    "    print(\"1. Sample size: \", sample_size)\n",
    "    print(\"2. Population mean:\", np.mean(table.column(label)))\n",
    "    print(\"3. Average of sample means: \", np.mean(means))\n",
    "    print(\"4. Population SD:\", population_std)\n",
    "    print(\"5. SD of sample means:\", np.std(means))\n",
    "    print(\"6. Population SD / sqrt(sample size)\", population_std / np.sqrt(sample_size), \"\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a97cad1-6671-46e4-b7d6-024c5c43f526",
   "metadata": {},
   "outputs": [],
   "source": [
    "for sample_size in range(100, 501, 200):\n",
    "    simulate_sample_mean(song_lengths, \"seconds\", sample_size, 10000)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f206fb0-d65a-4bff-85e3-5eee31cb91e6",
   "metadata": {},
   "source": [
    "What observations can you make from the above graphs regarding\n",
    "- The population mean and standard deviation (#2, #4)\n",
    "- The average of the sample means (#3)\n",
    "- The SD of the sample means (#5)\n",
    "- The ratio of the population SD to the square root of the sample size (#6)\n",
    "- The relationship between #5 and #6"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "437388b0-7b8d-4f4d-b95e-7d18ce52b0d0",
   "metadata": {},
   "source": [
    "**Statistics Theory** - If the sample size is fixed and the samples are drawn at random with replacement \n",
    "from the population, then\n",
    "\n",
    "          SD of all possible sample means = population SD / sqrt(sample size)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fddbeb7f-d64d-4604-b09c-0de942d60061",
   "metadata": {},
   "source": [
    "The SD of all possible sample means measures how variable the sample mean can be. *As such, \n",
    "it is a measure of the accuracy of the sample mean as an estimate of the population mean*. \n",
    "The smaller the SD, the more accurate the estimate.\n",
    "\n",
    "The formula shows that:\n",
    "- The population size doesn’t affect the accuracy of the sample mean. \n",
    "The population size doesn’t appear anywhere in the formula.\n",
    "- The population SD is a constant; it’s the same for every sample drawn from the population. The sample size can be varied. \n",
    "Because the sample size appears in the denominator, the variability of the sample mean decreases as the sample size increases, \n",
    "and hence the accuracy increases."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95e983ff-fbf1-4517-afce-72a12b0bf8f9",
   "metadata": {},
   "source": [
    "Thus in general, when you multiply the sample size by a factor, the accuracy of the sample mean goes up \n",
    "by the square root of that factor."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "275d982c-8e43-477e-9175-45e1a16cd76e",
   "metadata": {},
   "source": [
    "# Chapter 14.6 - Choosing a Sample Size"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ccce54c-8cc6-4380-9021-5fc13d2a1ea7",
   "metadata": {},
   "source": [
    "Suppose we want to sample the population of the common housefly to determine how often\n",
    "a certain proboscis shape appears.  If a 95% confidence interval is desired with an\n",
    "accuracy level of at least 10%, how many samples do we need?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d14c9db-b268-486f-8caa-76b15ce09cb3",
   "metadata": {},
   "source": [
    "The **Central Limit Theorem** tells us that for normally distributed variables, \n",
    "the interval “center +/- 2 SD\" contains 95% of the data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b8fecfc-2153-40a2-9edb-fea82cd1f528",
   "metadata": {},
   "source": [
    "Since it is reasonable to assume we are drawing from a population with replacement and the sample size will be fixed,\n",
    "we can plug into the formula from 14.5: \n",
    "\n",
    "        (4 * Population SD) / sqrt(sample size) <= .10\n",
    "\n",
    "**or equivalently:** \n",
    "\n",
    "        sample size >= (40 * Population SD) ** 2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0be3d044-bab2-478a-bb85-ea2c25b94ad1",
   "metadata": {},
   "source": [
    "**More Statistics Theory** - The Population SD can never be more than 0.5.\n",
    "\n",
    "Thus, we can guarantee our desired result if"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66fd9b2c-e499-495b-8e41-7cb3968ce8d6",
   "metadata": {},
   "source": [
    "        sample size >= (40 *.5) ** 2 \n",
    "        sample size >= 20 ** 2\n",
    "        sample size >= 400"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02b98c5f-b764-4730-adcf-ba18d5636683",
   "metadata": {},
   "source": [
    "**Test Your Understanding** - How large should the sample be if a 5% level of accuracy is desired?"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}