{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c68e4cb2-ff06-4e8c-b837-24606020902b",
   "metadata": {},
   "source": [
    "# 😵 相似性度量\n",
    "\n",
    "在机器学习领域，经常需要度量两组数据之间的相似度，这就涉及到相似度的度量方法。\n",
    "\n",
    "对于两个向量$𝑋 = (x_1, x_2, ..., x_n)$和$Y = (y_1, y_2, ..., y_n)$，有以下方法进行度量：\n",
    "\n",
    "1. 欧氏距离（Euclidean Distance）\n",
    "2. 曼哈顿距离（Manhattan Distance）\n",
    "3. 闵可夫斯基距离（Minkowski Distance）\n",
    "4. 余弦相似度（Cosine Similarity）\n",
    "5. 皮尔逊相关系数（Pearson Correlation Coefficient）\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "0bc323ba-9e6d-4ec6-881a-b7d2da28e4be",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-11-19T15:40:58.617200400Z",
     "start_time": "2023-11-19T15:40:58.276227600Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from scipy.spatial import distance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5b082235-1234-41c8-a4ec-def3c346dc2d",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-11-19T15:41:00.343779100Z",
     "start_time": "2023-11-19T15:41:00.317500300Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X = [2.08460537 4.81681062 4.20538035 8.59181999 1.71161554 3.38863961\n",
      " 2.70532833 6.9104135  2.20404517 8.11950921]\n",
      "Y = [0.10526874 5.61203697 8.13726187 7.45100296 1.89111355 0.06140866\n",
      " 7.72043871 9.57832168 7.01937884 2.97578267]\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(22)\n",
    "\n",
    "x = np.random.random(10) * 10\n",
    "y = np.random.random(10) * 10\n",
    "print(f'X = {x}\\nY = {y}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23d02539-68ce-4be7-8883-97c264767c19",
   "metadata": {},
   "source": [
    "### 1. 欧氏距离（Euclidean Distance）\n",
    "\n",
    "欧氏距离（L2范数）是计算两个向量之间的直线距离，即两个向量之间对应元素差的平方和的平方根。它在处理连续数据时常被使用。\n",
    "\n",
    "$$dist = \\sqrt{\\sum_{i=1}^n(x_i-y_i)^2}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "ebb62a35-582c-425a-a7ef-223dd0e4f6e8",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-11-18T17:19:04.249290Z",
     "start_time": "2023-11-18T17:19:04.245768100Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "欧氏距离 = 10.692447183086724\n",
      "L2范数 = 10.692447183086724\n",
      "Scipy欧式距离 = 10.692447183086724\n"
     ]
    }
   ],
   "source": [
    "def euclidean_dist(x, y):\n",
    "    return np.sqrt(np.sum(np.square(x - y)))\n",
    "\n",
    "\n",
    "dist = euclidean_dist(x, y)\n",
    "print(f'欧氏距离 = {dist}')\n",
    "dist = np.linalg.norm(x - y)\n",
    "print(f'L2范数 = {dist}')\n",
    "dist = distance.euclidean(x, y)\n",
    "print(f'Scipy欧式距离 = {dist}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "929fb5203384d831",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "### 2. 曼哈顿距离（Manhattan Distance）\n",
    "\n",
    "曼哈顿距离（也称城市街区距离、L1范数）是计算两个向量之间的城市街区距离，即两个向量之间对应元素差的绝对值之和。它在处理离散数据或稀疏数据时常被使用。\n",
    "\n",
    "$$dist = \\sum_{i=1}^n|x_i-y_i|$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "7fe06546-e817-4dbe-aa47-c23209527d47",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-11-18T17:19:06.705066200Z",
     "start_time": "2023-11-18T17:19:06.699545400Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "曼哈顿距离 = 28.99606925332795\n",
      "L1范数 = 28.99606925332795\n",
      "Scipy曼哈顿距离 = 28.99606925332795\n"
     ]
    }
   ],
   "source": [
    "def manhattan_dist(x, y):\n",
    "    return sum(np.abs(x - y))\n",
    "\n",
    "\n",
    "dist = manhattan_dist(x, y)\n",
    "print(f'曼哈顿距离 = {dist}')\n",
    "dist = np.linalg.norm(x - y, ord=1)\n",
    "print(f'L1范数 = {dist}')\n",
    "dist = distance.cityblock(x, y)\n",
    "print(f'Scipy曼哈顿距离 = {dist}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "686de903554c3b84",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "### 3. 闵可夫斯基距离（Minkowski Distance）\n",
    "\n",
    "闵可夫斯基距离是欧氏距离和曼哈顿距离的一种推广，可以通过调整参数来控制距离的计算方式。\n",
    "$$\n",
    "d(X, Y) = (|x_1-y_1|^p + |x_2-y_2|^p + ... + |x_n-y_n|^p)^{1/p}\n",
    "$$\n",
    "其中，$p$是控制距离计算方式的参数，当$p=2$时，退化为欧氏距离；当$p=1$时，退化为曼哈顿距离"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "c474efa4-4a0d-45eb-92d4-c65a48108b71",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-11-18T17:19:49.348689100Z",
     "start_time": "2023-11-18T17:19:49.336679600Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "闵可夫斯基距离 = 7.93828244203566\n",
      "Scipy闵可夫斯基距离 = 7.93828244203566\n"
     ]
    }
   ],
   "source": [
    "def minkowski_dist(x, y, p=1):\n",
    "    return np.linalg.norm(x - y, ord=p)\n",
    "\n",
    "\n",
    "p = 3\n",
    "dist = minkowski_dist(x, y, p=p)\n",
    "print(f'闵可夫斯基距离 = {dist}')\n",
    "dist = distance.minkowski(x, y, p=p)\n",
    "print(f'Scipy闵可夫斯基距离 = {dist}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "273b193a4beb4c3a",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "### 4. 余弦相似度（Cosine Similarity）\n",
    "\n",
    "余弦相似度衡量的是两个向量之间的夹角，而不是直线距离。它在处理文本数据、稀疏数据或高维数据时常被使用。\n",
    "\n",
    "$$dist = 1-\\frac{x \\cdot y}\n",
    "            {||x||_2 ||y||_2} \n",
    "$$\n",
    "$$ similarity= \\frac{\\Sigma_ix_iy_i}\n",
    "            {\\sqrt{\\Sigma_ix_i^2}\\sqrt{\\Sigma_iy_i^2}} \n",
    "$$\n",
    "\n",
    "其中，$X·Y$表示点积（内积），$||X||$和$||Y||$表示L2范数（长度）\n",
    "\n",
    "余弦相似度的取值范围在 -1 到 1 之间，值越接近 1 表示两个向量越相似，值越接近 -1 表示两个向量越不相似，值为 0 表示两个向量正交（无相似性）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "24d795ec-c1ac-4de0-aeef-8be223aebc71",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-11-18T17:34:31.526658Z",
     "start_time": "2023-11-18T17:34:31.513398300Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "余弦相似距离 = 0.017127813065678232, 余弦相似度 = 0.9828721869343218\n",
      "Scipy余弦相似距离 = 0.01712781306567812, 余弦相似度 = 0.9828721869343219\n"
     ]
    }
   ],
   "source": [
    "def cosine_similarity(x, y):\n",
    "    x = np.asarray(x)\n",
    "    y = np.asarray(y)\n",
    "    return np.dot(x, y) / (np.linalg.norm(x) * np.linalg.norm(y))\n",
    "\n",
    "\n",
    "sim = cosine_similarity(x, y)\n",
    "print(f'余弦相似距离 = {1 - sim}, 余弦相似度 = {sim}')\n",
    "dist = distance.cosine(x, y)\n",
    "print(f'Scipy余弦相似距离 = {dist}, 余弦相似度 = {1 - dist}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "568805f2d3428011",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "### 5. 皮尔逊相关系数（Pearson Correlation Coefficient）\n",
    "\n",
    "皮尔逊相关系数衡量的是两个向量之间的线性相关性，取值范围在-1到1之间。它在处理线性相关性问题时常被使用。\n",
    "\n",
    "$$dist =  1 - \\frac{(x - \\bar{x}) \\cdot (y - \\bar{y})}\n",
    "           {{||(x - \\bar{x})||}_2 {||(y - \\bar{y})||}_2}\n",
    "$$\n",
    "\n",
    "$$coor = \\frac{(x - \\bar{x}) \\cdot (y - \\bar{y})}\n",
    "           {{||(x - \\bar{x})||}_2 {||(y - \\bar{y})||}_2}\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "eeba3b1b-b2ee-4ea8-b8ab-853fff9b9de6",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-11-18T17:41:46.626073400Z",
     "start_time": "2023-11-18T17:41:46.581374400Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "皮尔逊距离 = 0.6357804085261578, 皮尔逊相关性 = 0.36421959147384214\n",
      "Scipy皮尔逊距离 = 0.6357804085261578, 皮尔逊相关性 = 0.3642195914738422\n"
     ]
    }
   ],
   "source": [
    "def pearson_r(x, y):\n",
    "    return np.corrcoef(x, y)[0, 1]\n",
    "\n",
    "\n",
    "def pearson_dist(x, y):\n",
    "    return 1 - np.corrcoef(x, y)[0, 1]\n",
    "\n",
    "\n",
    "dist = pearson_dist(x, y)\n",
    "r = pearson_r(x, y)\n",
    "print(f'皮尔逊距离 = {dist}, 皮尔逊相关性 = {r}')\n",
    "dist = distance.correlation(x, y)\n",
    "print(f'Scipy皮尔逊距离 = {dist}, 皮尔逊相关性 = {1 - dist}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab9fc5965ad7d115",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "### 6. 杰卡德相似度（Jaccard Similarity）\n",
    "\n",
    "杰卡德相似系数(Jaccard similarity coefficient)，也称杰卡德指数(Jaccard Index)，用于计算两个集合之间的相似性，它衡量的是两个集合交集的大小与并集的大小之间的比例。\n",
    "\n",
    "杰卡德距离(Jaccard Distance) 是用来衡量两个集合差异性的一种指标，它是杰卡德相似系数的补集，被定义为1减去Jaccard相似系数。\n",
    "\n",
    "$$dist = \\frac{c_{TF} + c_{FT}}\n",
    "        {c_{TT} + c_{FT} + c_{TF}}\n",
    "$$\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "03af84b0-7b16-4529-843b-103fde3c9bce",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-11-18T17:47:30.406643800Z",
     "start_time": "2023-11-18T17:47:30.403125700Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Scipy杰卡德距离 = 1.0, 杰卡德相关性 = 0.0\n"
     ]
    }
   ],
   "source": [
    "dist = distance.jaccard(x, y)\n",
    "print(f'Scipy杰卡德距离 = {dist}, 杰卡德相关性 = {1 - dist}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "909e133e81a1ccd6",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "### 7. 马氏距离（Mahalanobis Distance）\n",
    "\n",
    "马氏距离是一种用于衡量两个样本点之间的距离的度量方法。与欧氏距离不同，马氏距离考虑了数据的协方差结构，可以有效地处理特征之间的相关性。\n",
    "\n",
    "$$dist = \\sqrt{(x-y)^T\\Sigma^{-1}(x-y)}$$\n",
    "$$dist = \\sqrt{ (u-v) V^{-1} (u-v)^T } $$\n",
    "\n",
    "其中，(x-y)、(u-v)是样本点之间的差向量，$\\Sigma^{-1}$是协方差矩阵$\\Sigma$的逆矩阵，$T$表示向量的转置运算。\n",
    "\n",
    "马氏距离的计算考虑了特征之间的相关性，通过协方差矩阵$\\Sigma$的逆矩阵来对特征进行缩放和旋转，使得距离计算更加准确。当协方差矩阵$\\Sigma$为单位矩阵时，马氏距离退化为欧氏距离。\n",
    "\n",
    "马氏距离常用于聚类分析、异常检测和模式识别等领域，特别是在处理具有相关特征的数据时，能够更好地捕捉数据之间的真实距离。\n",
    "\n",
    "> 这里我对马氏距离的理解不是很深入，协方差矩阵的计算应该基于总体样本，样本量应该大于特征数，才能保证协方差矩阵逆矩阵存在"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "d71c9191-f944-4e58-902d-134f7078f14a",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-11-19T15:53:59.306697100Z",
     "start_time": "2023-11-19T15:53:59.249774200Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "马氏距离 = [[4.19164339]]\n",
      "Scipy马氏距离距离 = [[4.19164339]]\n"
     ]
    }
   ],
   "source": [
    "def mahalanobis_dist(x, y, inv_cov):\n",
    "    return np.sqrt(np.dot(np.dot(x-y, inv_cov), (x-y).T))\n",
    "\n",
    "\n",
    "a = np.random.random(10) * 10\n",
    "b = np.random.random(10) * 10\n",
    "c = np.random.random(10) * 10\n",
    "d = np.random.random(10) * 10\n",
    "e = np.random.random(10) * 10\n",
    "f = np.random.random(10) * 10\n",
    "g = np.random.random(10) * 10\n",
    "h = np.random.random(10) * 10\n",
    "mat = np.asarray([x, y, a, b, c, d, e, f, g, h]).T\n",
    "# 计算总体样本的协方差\n",
    "cov = np.cov(mat)\n",
    "# 计算协方差逆矩阵\n",
    "VI = np.linalg.inv(np.matrix(cov))\n",
    "# 计算马氏距离\n",
    "\n",
    "\n",
    "dist = mahalanobis_dist(x, y, VI)\n",
    "print(f'马氏距离 = {dist}')\n",
    "dist = distance.mahalanobis(x, y, VI)\n",
    "print(f'Scipy马氏距离距离 = {dist}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d48385da8715da9",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "### 8. 汉明距离\n",
    "\n",
    "两个等长字符串s1与s2之间的汉明距离定义为将其中一个变为另外一个所需要作的最小替换次数\n",
    "\n",
    "例如：1011101与1001001之间的汉明距离是2/7。\n",
    "\n",
    "$$ \\frac{c_{01} + c_{10}}{n} $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "c8f0534967498e52",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-11-19T16:00:52.678368300Z",
     "start_time": "2023-11-19T16:00:52.663848600Z"
    },
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "汉明距离: 0.6\n"
     ]
    }
   ],
   "source": [
    "x = np.array([0,1,0,1,0])\n",
    "y = np.array([1,1,1,1,1])\n",
    "print(\"汉明距离:\",distance.hamming(x,y))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8df4e5c4095a0ef9",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "## 参考\n",
    "\n",
    "[1] 知乎，@马东什么，[点积相似度、余弦相似度、欧几里得相似度\n",
    "](https://zhuanlan.zhihu.com/p/159244903)\n",
    "\n",
    "[2] CSDN，@川川1212，[协方差、协方差矩阵、马氏距离与欧式距离的理解\n",
    "](https://blog.csdn.net/chuanchuanmama/article/details/106938512)\n",
    "\n",
    "[3] CSDN，@lijfrank，[马氏距离-协方差矩阵\n",
    "](https://blog.csdn.net/Frank_LJiang/article/details/102686135)\n",
    "\n",
    "[4] 知乎，@Ph0en1x，[马氏距离(Mahalanobis Distance)\n",
    "](https://zhuanlan.zhihu.com/p/46626607)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}