{
"cells": [
{
"cell_type": "markdown",
"id": "a7a24702-6a96-4e8a-a7dd-d5e713a816d4",
"metadata": {},
"source": [
"# 🐞 加利福尼亚房价数据集"
]
},
{
"cell_type": "markdown",
"id": "f768f168c69abb3f",
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"source": [
"## 壹丨数据集简介\n",
"\n",
"简介：包含平均房屋价值作为目标变量，输入变量包括：平均收入、住房平均年龄、平均房间、平均卧室、人口、平均职业、纬度、精度[^1]\n",
"\n",
"原始数据可以从StatLib获取：http://lib.stat.cmu.edu/datasets/\n",
"\n",
"数据量：[20640, 9]"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "b0ed9c34db5e5ed0",
"metadata": {
"ExecuteTime": {
"end_time": "2023-11-11T14:14:11.683001700Z",
"start_time": "2023-11-11T14:14:11.332591700Z"
},
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"dict_keys(['data', 'target', 'frame', 'target_names', 'feature_names', 'DESCR'])\n"
]
}
],
"source": [
"# 导入加利福尼亚房价数据\n",
"from sklearn.datasets import fetch_california_housing\n",
"from sklearn.model_selection import train_test_split\n",
"\n",
"data = fetch_california_housing()\n",
"print(data.keys())\n",
"\n",
"# 划分数据集\n",
"x = data['data']\n",
"y = data['target']\n",
"train_x, test_x, train_y, test_y = train_test_split(x, y, test_size=0.3, random_state=22)"
]
},
{
"cell_type": "markdown",
"id": "48923793f659802d",
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"source": [
"## 贰丨scikit-learn调用"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "3b4d43981b12ae1b",
"metadata": {
"ExecuteTime": {
"end_time": "2023-11-11T14:14:16.657249800Z",
"start_time": "2023-11-11T14:14:16.611811800Z"
},
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [],
"source": [
"from sklearn.datasets import fetch_california_housing\n",
"\n",
"# 载入数据\n",
"california_housing = fetch_california_housing(as_frame=True)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "60f946105606c2c7",
"metadata": {
"ExecuteTime": {
"end_time": "2023-11-11T14:14:18.398503200Z",
"start_time": "2023-11-11T14:14:18.374610300Z"
},
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
".. _california_housing_dataset:\n",
"\n",
"California Housing dataset\n",
"--------------------------\n",
"\n",
"**Data Set Characteristics:**\n",
"\n",
" :Number of Instances: 20640\n",
"\n",
" :Number of Attributes: 8 numeric, predictive attributes and the target\n",
"\n",
" :Attribute Information:\n",
" - MedInc median income in block group\n",
" - HouseAge median house age in block group\n",
" - AveRooms average number of rooms per household\n",
" - AveBedrms average number of bedrooms per household\n",
" - Population block group population\n",
" - AveOccup average number of household members\n",
" - Latitude block group latitude\n",
" - Longitude block group longitude\n",
"\n",
" :Missing Attribute Values: None\n",
"\n",
"This dataset was obtained from the StatLib repository.\n",
"https://www.dcc.fc.up.pt/~ltorgo/Regression/cal_housing.html\n",
"\n",
"The target variable is the median house value for California districts,\n",
"expressed in hundreds of thousands of dollars ($100,000).\n",
"\n",
"This dataset was derived from the 1990 U.S. census, using one row per census\n",
"block group. A block group is the smallest geographical unit for which the U.S.\n",
"Census Bureau publishes sample data (a block group typically has a population\n",
"of 600 to 3,000 people).\n",
"\n",
"A household is a group of people residing within a home. Since the average\n",
"number of rooms and bedrooms in this dataset are provided per household, these\n",
"columns may take surprisingly large values for block groups with few households\n",
"and many empty houses, such as vacation resorts.\n",
"\n",
"It can be downloaded/loaded using the\n",
":func:`sklearn.datasets.fetch_california_housing` function.\n",
"\n",
".. topic:: References\n",
"\n",
" - Pace, R. Kelley and Ronald Barry, Sparse Spatial Autoregressions,\n",
" Statistics and Probability Letters, 33 (1997) 291-297\n"
]
}
],
"source": [
"# 查看数据集描述\n",
"print(california_housing.DESCR)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "493be7273d5d279b",
"metadata": {
"ExecuteTime": {
"end_time": "2023-11-11T14:14:21.129418400Z",
"start_time": "2023-11-11T14:14:21.062336200Z"
},
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" MedInc | \n",
" HouseAge | \n",
" AveRooms | \n",
" AveBedrms | \n",
" Population | \n",
" AveOccup | \n",
" Latitude | \n",
" Longitude | \n",
" MedHouseVal | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" 8.3252 | \n",
" 41.0 | \n",
" 6.984127 | \n",
" 1.023810 | \n",
" 322.0 | \n",
" 2.555556 | \n",
" 37.88 | \n",
" -122.23 | \n",
" 4.526 | \n",
"
\n",
" \n",
" | 1 | \n",
" 8.3014 | \n",
" 21.0 | \n",
" 6.238137 | \n",
" 0.971880 | \n",
" 2401.0 | \n",
" 2.109842 | \n",
" 37.86 | \n",
" -122.22 | \n",
" 3.585 | \n",
"
\n",
" \n",
" | 2 | \n",
" 7.2574 | \n",
" 52.0 | \n",
" 8.288136 | \n",
" 1.073446 | \n",
" 496.0 | \n",
" 2.802260 | \n",
" 37.85 | \n",
" -122.24 | \n",
" 3.521 | \n",
"
\n",
" \n",
" | 3 | \n",
" 5.6431 | \n",
" 52.0 | \n",
" 5.817352 | \n",
" 1.073059 | \n",
" 558.0 | \n",
" 2.547945 | \n",
" 37.85 | \n",
" -122.25 | \n",
" 3.413 | \n",
"
\n",
" \n",
" | 4 | \n",
" 3.8462 | \n",
" 52.0 | \n",
" 6.281853 | \n",
" 1.081081 | \n",
" 565.0 | \n",
" 2.181467 | \n",
" 37.85 | \n",
" -122.25 | \n",
" 3.422 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" MedInc | \n",
" HouseAge | \n",
" AveRooms | \n",
" AveBedrms | \n",
" Population | \n",
" AveOccup | \n",
" Latitude | \n",
" Longitude | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" 8.3252 | \n",
" 41.0 | \n",
" 6.984127 | \n",
" 1.023810 | \n",
" 322.0 | \n",
" 2.555556 | \n",
" 37.88 | \n",
" -122.23 | \n",
"
\n",
" \n",
" | 1 | \n",
" 8.3014 | \n",
" 21.0 | \n",
" 6.238137 | \n",
" 0.971880 | \n",
" 2401.0 | \n",
" 2.109842 | \n",
" 37.86 | \n",
" -122.22 | \n",
"
\n",
" \n",
" | 2 | \n",
" 7.2574 | \n",
" 52.0 | \n",
" 8.288136 | \n",
" 1.073446 | \n",
" 496.0 | \n",
" 2.802260 | \n",
" 37.85 | \n",
" -122.24 | \n",
"
\n",
" \n",
" | 3 | \n",
" 5.6431 | \n",
" 52.0 | \n",
" 5.817352 | \n",
" 1.073059 | \n",
" 558.0 | \n",
" 2.547945 | \n",
" 37.85 | \n",
" -122.25 | \n",
"
\n",
" \n",
" | 4 | \n",
" 3.8462 | \n",
" 52.0 | \n",
" 6.281853 | \n",
" 1.081081 | \n",
" 565.0 | \n",
" 2.181467 | \n",
" 37.85 | \n",
" -122.25 | \n",
"
\n",
" \n",
"
\n",
"
"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import matplotlib.pyplot as plt\n",
"from PyAqua import config\n",
"config.plt_style()\n",
"\n",
"# 查看每个特征的分布\n",
"california_housing.frame.hist(figsize=(10, 9), bins=30, edgecolor=\"black\")\n",
"plt.subplots_adjust(hspace=0.7, wspace=0.4)"
]
},
{
"cell_type": "markdown",
"id": "b83102eca2502f9f",
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"source": [
"平均房间、平均卧室、平均职业和人口明显的存在离群值，可以大概看下数据的统计结果"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "ef96ad4f78b7bcb1",
"metadata": {
"ExecuteTime": {
"end_time": "2023-11-11T14:17:34.075625Z",
"start_time": "2023-11-11T14:17:34.011403800Z"
},
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" AveRooms | \n",
" AveBedrms | \n",
" AveOccup | \n",
" Population | \n",
"
\n",
" \n",
" \n",
" \n",
" | count | \n",
" 20640.000000 | \n",
" 20640.000000 | \n",
" 20640.000000 | \n",
" 20640.000000 | \n",
"
\n",
" \n",
" | mean | \n",
" 5.429000 | \n",
" 1.096675 | \n",
" 3.070655 | \n",
" 1425.476744 | \n",
"
\n",
" \n",
" | std | \n",
" 2.474173 | \n",
" 0.473911 | \n",
" 10.386050 | \n",
" 1132.462122 | \n",
"
\n",
" \n",
" | min | \n",
" 0.846154 | \n",
" 0.333333 | \n",
" 0.692308 | \n",
" 3.000000 | \n",
"
\n",
" \n",
" | 25% | \n",
" 4.440716 | \n",
" 1.006079 | \n",
" 2.429741 | \n",
" 787.000000 | \n",
"
\n",
" \n",
" | 50% | \n",
" 5.229129 | \n",
" 1.048780 | \n",
" 2.818116 | \n",
" 1166.000000 | \n",
"
\n",
" \n",
" | 75% | \n",
" 6.052381 | \n",
" 1.099526 | \n",
" 3.282261 | \n",
" 1725.000000 | \n",
"
\n",
" \n",
" | max | \n",
" 141.909091 | \n",
" 34.066667 | \n",
" 1243.333333 | \n",
" 35682.000000 | \n",
"
\n",
" \n",
"
\n",
"