5Coming from Pandas.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 区别"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 区别1，polars没有索引"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pandas gives a label to each row with an index.   \n",
    "Polars does not use an index and  \n",
    " each row **is indexed by its integer position** in the table."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 区别2，polars使用arrow array"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Polars uses Apache Arrow arrays to represent data in memory  \n",
    " while Pandas uses Numpy arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Apache Arrow is an emerging standard for in-memory columnar analytics that can   - -\n",
    "- accelerate data load times, \n",
    "- reduce memory usage \n",
    "- and accelerate calculations.\n",
    "Polars can convert data to Numpy format with the `to_numpy` method."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "即arrow array更快，而且polars也支持向numpy转换"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 区别3，更多的并行支持"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Polars has more support for parallel operations than Pandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Polars exploits the strong support for concurrency in Rust to run many operations in parallel.   \n",
    "While some operations in Pandas are multi-threaded the core of the library is single-threaded   \n",
    "and an additional library such as **Dask** must be used to parallelise operations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "即pandas是单线程的，不得不用类似于dask之类的包来并行加速"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 区别4 polars有lazy模式"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Polars can lazily evaluate queries and apply query optimization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Eager evaluation is where code is evaluated as soon as you run the code.  \n",
    "Lazy evaluation is where running a line of code means that  \n",
    "    the underlying logic is added to a query plan rather than being evaluated.  \n",
    "\n",
    "Polars supports eager evaluation and lazy evaluation whereas Pandas only supports eager evaluation.   \n",
    "The lazy evaluation mode is powerful because Polars carries out automatic query optimization where it examines the query plan and looks for ways to accelerate the query or reduce memory usage.\n",
    "\n",
    "Dask also supports lazy evaluation where it generates a query plan. However, Dask does not carry out query optimization on the query plan.  \n",
    "Dask又被踩了一脚，即它虽然有lazy模式，但是他不做算法优化，只是延迟处理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 一些操作"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 用polars写法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "不要用pandas的写法来写polars，那样会很慢"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在pandas中，选择列a\n",
    "```python\n",
    "df['a']\n",
    "df.loc[:,'a']\n",
    "```\n",
    "\n",
    "在polars中，选择列a\n",
    "```python\n",
    "df.select(['a'])\n",
    "```\n",
    "\n",
    "在polars中，选择列a中满足条件的行\n",
    "```python\n",
    "df.filter(pl.col('a')<10)\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Be lazy，多使用懒模式"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Working in lazy evaluation mode is straightforward and  \n",
    " should be your default in Polars as the lazy mode allows Polars to do query optimization.\n",
    "\n",
    "We can run in lazy mode by either using an implicitly lazy function (such as **scan_csv**) or explicitly using the lazy method.\n",
    "\n",
    "Take the following simple example where we read a CSV file from disk   and do a groupby.   \n",
    "The CSV file has numerous columns but we just want to do a groupby on one of the id columns (id1)   \n",
    "and then sum by a value column (v1). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Pandas this would be:\n",
    "\n",
    "```python\n",
    "df = pd.read_csv(csvFile)\n",
    "groupedDf = df.loc[:,['id1','v1']].groupby('id1').sum('v1')\n",
    "```\n",
    "1 读入  \n",
    "2 选出对应的数据，然后按id1分组，按v1求和"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Polars you can build this query in lazy mode with query optimization   \n",
    "and evaluate it by replacing the eager Pandas function read_csv with\n",
    " the implicitly lazy Polars function scan_csv:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Polars this would be:\n",
    "\n",
    "```python\n",
    "df = pl.scan_csv(csvFile)\n",
    "roupedDf = df.groupby('id1').agg(pl.col('v1').sum()).collect()\n",
    "```\n",
    "1 读入  \n",
    "2 选出对应的数据，然后按id1分组，按v1求和"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Polars optimizes this query by identifying that   \n",
    "**only the id1 and v1 columns are relevant and**  \n",
    " **so will only read these columns from the CSV.**  \n",
    "By calling the `.collect` method at the end of the second line we instruct Polars to eagerly evaluate the query.\n",
    "\n",
    "If you do want to run this query in eager mode you can just replace scan_csv with read_csv in the Polars code.\n",
    "\n",
    "Read more about working with lazy evaluation in the lazy API section."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Express yourself，多用表达式"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A typical Pandas script consists of multiple data transformations that are executed sequentially.   \n",
    "However, in Polars these transformations can be executed in parallel using expressions.  \n",
    "即pandas是序列地执行一系列操作（他的调用也是链式地）\n",
    "而polars是并行的运行表达式"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Column assignment，列赋值/列新增"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have a dataframe df with a column called `value`.  \n",
    "We want to add two new columns,   \n",
    "a column called `tenXValue` where the value column is multiplied by 10 and  \n",
    "a column called `hundredXValue` where the value column is multiplied by 100."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Pandas this would be:\n",
    "\n",
    "```python\n",
    "df[\"tenXValue\"] = df[\"value\"] * 10\n",
    "df[\"hundredXValue\"] = df[\"value\"] * 100\n",
    "```\n",
    "These column assignments are executed sequentially."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Polars we add columns to df using the `.with_column` method   \n",
    "and name them with the .alias method:\n",
    "\n",
    "```python\n",
    "df.with_columns([\n",
    "    (pl.col(\"value\") * 10).alias(\"tenXValue\"),\n",
    "    (pl.col(\"value\") * 100).alias(\"hundredXValue\"),\n",
    "])\n",
    "```\n",
    "These column assignments are executed **in parallel**.  \n",
    "所以破拉绒的语法看起来更费劲，但他是并行的"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Column assignment based on predicate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case we have a dataframe df with columns `a`,`b` and `c`.   \n",
    "We want to re-assign the values in column `a` based on a condition.   \n",
    "When the value in column `c` is equal to 2 then we replace the value in `a` with the value in `b`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Pandas this would be:\n",
    "```python\n",
    "df.loc[df[\"c\"] == 2, \"a\"] = df.loc[df[\"c\"] == 2, \"b\"]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Polars this would be:\n",
    "\n",
    "```python\n",
    "df.with_column(\n",
    "    pl.when(pl.col(\"c\") == 2)\n",
    "    .then(pl.col(\"b\"))\n",
    "    .otherwise(pl.col(\"a\")).alias(\"a\")\n",
    ")\n",
    "```\n",
    "仔细对比下两种操作逻辑，很有意思  \n",
    "pandas是对满足条件的a修改为的值  \n",
    "polars是，如果满足条件，则值取b的，不满足条件，值取a的  \n",
    "那么自然就不会对原来的a进行修改了"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Polars way is pure in that **the original DataFrame is not modified**.  \n",
    "The mask is also not computed twice as in Pandas  \n",
    " (you could prevent this in Pandas, but that would require setting a temporary variable).\n",
    "也就是说，不会修改原df\n",
    "\n",
    "还跟对if语句并行\n",
    "Additionally Polars can compute every branch of   \n",
    "an `if -> then -> otherwise` in parallel.   \n",
    "This is valuable, when the branches get more expensive to compute."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Filtering"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We want to filter the dataframe df with housing data **based on some criteria**.  \n",
    "In Pandas you filter the dataframe by passing **Boolean expressions** to the loc method:\n",
    "```python\n",
    "df.loc[(df['sqft_living'] > 2500) & (df['price'] < 300000)]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Polars this would be:\n",
    "\n",
    "```python\n",
    "df.filter(\n",
    "    (pl.col(\"m2_living\") > 2500) & (pl.col(\"price\") < 300000)\n",
    ")\n",
    "```\n",
    "The query optimizer in Polars can also detect if you write multiple filters separately and combine them into a single filter in the optimized plan."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pandas transform"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Pandas documentation demonstrates an operation on a groupby called transform.  \n",
    "In this case we have a dataframe `df` and  \n",
    "we want **a new column showing the number of rows** in each group."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Pandas this would be:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>type</th>\n",
       "      <th>c</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>m</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>n</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>o</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>m</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>m</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>n</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>n</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  type  c\n",
       "0    m  1\n",
       "1    n  1\n",
       "2    o  1\n",
       "3    m  2\n",
       "4    m  2\n",
       "5    n  2\n",
       "6    n  2"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "df = pd.DataFrame({\n",
    "    \"type\": [\"m\", \"n\", \"o\", \"m\", \"m\", \"n\", \"n\"],\n",
    "    \"c\": [1, 1, 1, 2, 2, 2, 2],\n",
    "})\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>type</th>\n",
       "      <th>c</th>\n",
       "      <th>size</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>m</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>n</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>o</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>m</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>m</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>n</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>n</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  type  c  size\n",
       "0    m  1     3\n",
       "1    n  1     3\n",
       "2    o  1     3\n",
       "3    m  2     4\n",
       "4    m  2     4\n",
       "5    n  2     4\n",
       "6    n  2     4"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"size\"] = df.groupby(\"c\")[\"type\"].transform(len)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here Pandas \n",
    "1. does a groupby on \"c\",   \n",
    "2. takes column \"type\",   \n",
    "3. computes the group length and  \n",
    "4. then joins the result back to the original DataFrame producing:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "   c type size\n",
    "0  1    m    3\n",
    "1  1    n    3\n",
    "2  1    o    3\n",
    "3  2    m    4\n",
    "4  2    m    4\n",
    "5  2    n    4\n",
    "6  2    n    4\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Polars this would be:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "import polars as pl\n",
    "\n",
    "df.select([\n",
    "    pl.all(),\n",
    "    pl.col(\"type\").count().over(\"c\").alias(\"size\")\n",
    "])\n",
    "```\n",
    "说一下差异吧  \n",
    "panda的步骤上面讲了  \n",
    "那么polars呢  \n",
    "是按c分组————over(\"c\")  \n",
    "然后选出type列，然后统计其组内长度count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "输出\n",
    "```python\n",
    "shape: (7, 3)\n",
    "┌─────┬──────┬──────┐\n",
    "│ c   ┆ type ┆ size │\n",
    "│ --- ┆ ---  ┆ ---  │\n",
    "│ i64 ┆ str  ┆ u32  │\n",
    "╞═════╪══════╪══════╡\n",
    "│ 1   ┆ m    ┆ 3    │\n",
    "├╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌┤\n",
    "│ 1   ┆ n    ┆ 3    │\n",
    "├╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌┤\n",
    "│ 1   ┆ o    ┆ 3    │\n",
    "├╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌┤\n",
    "│ 2   ┆ m    ┆ 4    │\n",
    "├╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌┤\n",
    "│ 2   ┆ m    ┆ 4    │\n",
    "├╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌┤\n",
    "│ 2   ┆ n    ┆ 4    │\n",
    "├╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌┤\n",
    "│ 2   ┆ n    ┆ 4    │\n",
    "└─────┴──────┴──────┘\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Because we can store the whole operation in a single expression,  \n",
    "we can combine several window functions and even combine different groups!  \n",
    "因为会缓存所有的操作到一个表达式  \n",
    "所以多个窗口函数和分组都是可以结合的，这样就提升效率了  \n",
    "不过说实话，这个也就是手册的例子，我们一般也不会同时对多个组分组  \n",
    "然后又把他们弄到一起，一般还是分开弄然后merge吧？\n",
    "\n",
    "Polars will cache（缓存） window expressions that are applied over the same group,  \n",
    "so storing them in a single select is both convenient and optimal.\n",
    "In the following example we look at a case \n",
    "where we are calculating group statistics over \"c\" twice:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "df.select([\n",
    "    pl.all(),\n",
    "    pl.col(\"c\").count().over(\"c\").alias(\"size\"),\n",
    "    pl.col(\"c\").sum().over(\"type\").alias(\"sum\"),\n",
    "    pl.col(\"c\").reverse().over(\"c\").flatten().alias(\"reverse_type\")\n",
    "])\n",
    "\n",
    "# 输出\n",
    "shape: (7, 5)\n",
    "┌─────┬──────┬──────┬─────┬──────────────┐\n",
    "│ c   ┆ type ┆ size ┆ sum ┆ reverse_type │\n",
    "│ --- ┆ ---  ┆ ---  ┆ --- ┆ ---          │\n",
    "│ i64 ┆ str  ┆ u32  ┆ i64 ┆ i64          │\n",
    "╞═════╪══════╪══════╪═════╪══════════════╡\n",
    "│ 1   ┆ m    ┆ 3    ┆ 5   ┆ 2            │\n",
    "├╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤\n",
    "│ 1   ┆ n    ┆ 3    ┆ 5   ┆ 2            │\n",
    "├╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤\n",
    "│ 1   ┆ o    ┆ 3    ┆ 1   ┆ 2            │\n",
    "├╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤\n",
    "│ 2   ┆ m    ┆ 4    ┆ 5   ┆ 2            │\n",
    "├╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤\n",
    "│ 2   ┆ m    ┆ 4    ┆ 5   ┆ 1            │\n",
    "├╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤\n",
    "│ 2   ┆ n    ┆ 4    ┆ 5   ┆ 1            │\n",
    "├╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌╌┼╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤\n",
    "│ 2   ┆ n    ┆ 4    ┆ 5   ┆ 1            │\n",
    "└─────┴──────┴──────┴─────┴──────────────┘\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Missing data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pandas uses `NaN` and/or `None` values to indicate missing values \n",
    "    depending on the dtype of the column.  \n",
    "In addition the behaviour in Pandas varies depending on \n",
    "    whether the default dtypes or optional nullable arrays are used.  \n",
    "In Polars missing data corresponds to a `null` value for all data types.\n",
    "也就是说，pandas的缺失值，跟原数据类型有关，用不同的符号表示  \n",
    "但是破拉绒就是用null\n",
    "\n",
    "但是polars对于浮点型的缺失值，允许使用`NaN`\n",
    "For float columns Polars permits the use of `NaN` values.   \n",
    "These `NaN` values are not considered to be missing data but instead a special floating point value.\n",
    "\n",
    "In Pandas an integer column with missing values is cast to be a float column with `NaN` values for the missing values   \n",
    "(unless using optional nullable integer dtypes).   \n",
    "In Polars any missing values in an integer column are simply `null` values   \n",
    "and the column remains an integer column.\n",
    "\n",
    "See the missing data section for more details."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.13 ('base')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "8a4dcfaef72816f569ec4f1c2cac54e5fc489caf6a777ef6037521028c4ea056"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}