像Excel一样使用SQL进行数据分析

重庆鲜面店刘永菊 發表於 2023-6-11 00:00:00

Excel是数据分析中最常用的工具，利用Excel可以完成数据清洗，预处理，以及最常见的数据分类，数据筛选，分类汇总，以及数据透视等操作，而这些操作用SQL一样可以实现。SQL不仅可以从数据库中读取数据，还能通过不同的SQL函数语句直接返回所需要的结果，从而大大提高了自己在客户端应用程序中计算的效率。

<img title="像Excel一样使用SQL进行数据分析" alt="像Excel一样使用SQL进行数据分析" border="0" src="https://zhuji.jb51.net/uploads/img/202305/13d1f360828b18b14578de9b191c025f.jpg">
<h3>
1 重复数据处理</h3>

查找重复记录
<ol class="dp-sql">
<li class="alt">
SELECT * FROM user 
</li>
<li>
Where (nick_name,password) in 
</li>
<li class="alt">
( 
</li>
<li>
SELECT nick_name,password 
</li>
<li class="alt">
FROM user 
</li>
<li>
group by nick_name,password 
</li>
<li class="alt">
having count(nick_name)>1 
</li>
<li>
); 
</li>
</ol>

查找去重记录

查找id最大的记录
<ol class="dp-sql">
<li class="alt">
SELECT * FROM user 
</li>
<li>
WHERE id in 
</li>
<li class="alt">
(SELECT max(id) FROM user 
</li>
<li>
group by nick_name,password 
</li>
<li class="alt">
having count(nick_name)>1 
</li>
<li>
); 
</li>
</ol>

删除重复记录

只保留id值最小的记录
<ol class="dp-sql">
<li class="alt">
DELETE c1 
</li>
<li>
FROM customer c1,customer c2 
</li>
<li class="alt">
WHERE c1.cust_email=c2.cust_email 
</li>
<li>
AND c1.id>c2.id; 
</li>
<li class="alt">
DELETE FROM user Where (nick_name,password) in 
</li>
<li>
(SELECT nick_name,password FROM 
</li>
<li class="alt">
(SELECT nick_name,password FROM user 
</li>
<li>
group by nick_name,password 
</li>
<li class="alt">
having count(nick_name)>1) as tmp1 
</li>
<li>
) 
</li>
<li class="alt">
and id not in 
</li>
<li>
(SELECT id FROM 
</li>
<li class="alt">
(SELECT min(id) id FROM user 
</li>
<li>
group by nick_name,password 
</li>
<li class="alt">
having count(nick_name)>1) as tmp2 
</li>
<li>
); 
</li>
</ol>
<h3>
2 缺失值处理</h3>

查找缺失值记录
<ol class="dp-sql">
<li class="alt">
SELECT * FROM customer 
</li>
<li>
WHERE cust_email IS NULL; 
</li>
</ol>

更新列填充空值
<ol class="dp-sql">
<li class="alt">
UPDATE sale set city = "未知" 
</li>
<li>
WHERE city IS NULL; 
</li>
<li class="alt">
</li>
<li>
UPDATE orderitems set 
</li>
<li class="alt">
price_new=IFNULL(price_new,5.74); 
</li>
</ol>

查询并填充空值列
<ol class="dp-sql">
<li class="alt">
SELECT AVG(price_new) FROM orderitems; 
</li>
<li>
</li>
<li class="alt">
SELECT IFNULL(price_new,5.74) AS bus_ifnull 
</li>
<li>
FROM orderitems; 
</li>
</ol>
<h3>
3 计算列</h3>

更新表添加计算列
<ol class="dp-sql">
<li class="alt">
ALTER TABLE orderitems ADD price_new DECIMAL(8,2) NOT NULL; 
</li>
<li>
</li>
<li class="alt">
UPDATE orderitems set price_new= item_price*count; 
</li>
</ol>

查询计算列
<ol class="dp-sql">
<li class="alt">
SELECT item_price*count as sales FROM orderitems; 
</li>
</ol>
<h3>
4 排序</h3>

多列排序
<ol class="dp-sql">
<li class="alt">
SELECT * FROM orderitems 
</li>
<li>
ORDER BY price_new DESC,quantity; 
</li>
</ol>

查询排名前几的记录
<ol class="dp-sql">
<li class="alt">
SELECT * FROM orderitems 
</li>
<li>
ORDER BY price_new DESC LIMIT 5; 
</li>
</ol>

查询第10大的值
<ol class="dp-sql">
<li class="alt">
SELECT DISTINCT price_new 
</li>
<li>
FROM orderitems 
</li>
<li class="alt">
ORDER BY price_new DESC LIMIT 9,1; 
</li>
</ol>

排名

数值相同的排名相同且排名连续
<ol class="dp-sql">
<li class="alt">
SELECT prod_price, 
</li>
<li>
(SELECT COUNT(DISTINCT prod_price) 
</li>
<li class="alt">
FROM products 
</li>
<li>
WHERE prod_price>=a.prod_price 
</li>
<li class="alt">
) AS rank 
</li>
<li>
FROM products AS a 
</li>
<li class="alt">
ORDER BY rank ; 
</li>
</ol>
<h3>
5 字符串处理</h3>

字符串替换
<ol class="dp-sql">
<li class="alt">
UPDATE data1 SET city=REPLACE(city,'SH','shanghai'); 
</li>
<li>
</li>
<li class="alt">
SELECT city FROM data1; 
</li>
</ol>

按位置字符串截取

字符串截取可用于数据分列

MySQL 字符串截取函数：left(), right(), substring(), substring_index()
<ol class="dp-sql">
<li class="alt">
SELECT left('example.com', 3); 
</li>
</ol>

从字符串的第 4 个字符位置开始取，直到结束
<ol class="dp-sql">
<li class="alt">
SELECT substring('example.com', 4); 
</li>
</ol>

从字符串的第 4 个字符位置开始取，只取 2 个字符
<ol class="dp-sql">
<li class="alt">
SELECT substring('example.com', 4, 2); 
</li>
</ol>

按关键字截取字符串

取第一个分隔符之前的所有字符，结果是www
<ol class="dp-sql">
<li class="alt">
SELECT substring_index('www.google.com','.',1); 
</li>
</ol>

取倒数第二个分隔符之后的所有字符，结果是google.com;
<ol class="dp-sql">
<li class="alt">
SELECT substring_index('www.google.com','.',-2); 
</li>
</ol>
<h3>
6 筛选</h3>

通过操作符实现高级筛选

使用 AND OR IN NOT 等操作符实现高级筛选过滤
<ol class="dp-sql">
<li class="alt">
SELECT prod_name，prod_price FROM Products 
</li>
<li>
WHERE vend_id IN('DLL01','BRS01'); 
</li>
<li class="alt">
SELECT prod_name FROM Products WHERE NOT vend_id='DLL01'; 
</li>
</ol>

通配符筛选

常用通配符有% _ [] ^
<ol class="dp-sql">
<li class="alt">
SELECT * from customers WHERE country LIKE "CH%"; 
</li>
</ol>
<h3>
7 表联结</h3>

SQL表连接可以实现类似于Excel中的Vlookup函数的功能
<ol class="dp-sql">
<li class="alt">
SELECT vend_id,prod_name,prod_price 
</li>
<li>
FROM Vendors INNER JOIN Products 
</li>
<li class="alt">
ON Vendors.vend_id=Products.vend_id; 
</li>
<li>
</li>
<li class="alt">
SELECT prod_name,vend_name,prod_price,quantity 
</li>
<li>
FROM OderItems,Products,Vendors 
</li>
<li class="alt">
WHERE Products.vend_id=Vendors.vend_id 
</li>
<li>
AND OrderItems.prod_id=Products.prod_id 
</li>
<li class="alt">
AND order_num=20007; 
</li>
</ol>

自联结在一条SELECT语句中多次使用相同的表
<ol class="dp-sql">
<li class="alt">
SELECT c1.cust_od,c1.cust_name,c1.cust_contact 
</li>
<li>
FROM Customers as c1,Customers as c2 
</li>
<li class="alt">
WHERE c1.cust_name=c2.cust_name 
</li>
<li>
AND c2.cust_contact='Jim Jones'; 
</li>
</ol>
<h3>
8 数据透视</h3>

数据分组可以实现Excel中数据透视表的功能

数据分组

group by 用于数据分组 having 用于分组后数据的过滤
<ol class="dp-sql">
<li class="alt">
SELECT order_num,COUNT(*) as items 
</li>
<li>
FROM OrderItems 
</li>
<li class="alt">
GROUP BY order_num HAVING COUNT(*)>=3; 
</li>
</ol>

交叉表

通过CASE WHEN函数实现
<ol class="dp-sql">
<li class="alt">
SELECT data1.city, 
</li>
<li>
CASE WHEN colour = "A" THEN price END AS A, 
</li>
<li class="alt">
CASE WHEN colour = "B" THEN price END AS B, 
</li>
<li>
CASE WHEN colour = "C" THEN price END AS C, 
</li>
<li class="alt">
CASE WHEN colour = "F" THEN price END AS F 
</li>
<li>
FROM data1 
</li>
</ol>

原文链接：https://mp.weixin.qq.com/s?__biz=MzA3MTg4NjY4Mw==&mid=2457324446&idx=2&sn=9cbf3a9184e66e115c867a3326ca88e9&chksm=88a5d9aabfd250bc3f12aca480686547379692d25efc1e3b223670f27c1a8473f2329729dc9f&mpshare=1&

頁: [1]

琼殿技术论坛's Archiver

像Excel一样使用SQL进行数据分析