-
Notifications
You must be signed in to change notification settings - Fork 22
/
Copy pathupset.html
99 lines (96 loc) · 5.37 KB
/
upset.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
<html>
<meta content="width=device-width, initial-scale=1.0" name="viewport">
<head>
<title>
leontrolski - UpSet
</title>
<style>
body {margin: 5% auto; background: #fff7f7; color: #444444; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif; font-size: 16px; line-height: 1.8; max-width: 63%;}
@media screen and (max-width: 800px) {body {font-size: 14px; line-height: 1.4; max-width: 90%;}}
pre {width: 100%; border-top: 3px solid gray; border-bottom: 3px solid gray;}
a {border-bottom: 1px solid #444444; color: #444444; text-decoration: none; text-shadow: 0 1px 0 #ffffff; }
a:hover {border-bottom: 0;}
.inline {background: #b3b2b226; padding-left: 0.3em; padding-right: 0.3em; white-space: nowrap;}
blockquote {font-style: italic;color:black;background-color:#f2f2f2;padding:2em;}
details {border-bottom:solid 5px gray;}
img{max-width:30rem;}
.overflow{overflow-x: scroll;max-width:30rem;box-shadow:0 0 5px 0 gray;}
</style>
<link href="https://unpkg.com/[email protected]/themes/prism-vs.css" rel="stylesheet">
<script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.20.0/components/prism-core.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.20.0/plugins/autoloader/prism-autoloader.min.js"></script>
</head>
<body>
<a href="index.html">
<img src="pic.png" style="height:2em">
⇦
</a>
<p><i>2024-10-17</i></p>
<h1>UpSet plots, succinctly</h1>
<p>
<a href="https://en.wikipedia.org/wiki/UpSet_Plot">UpSet plots</a> are not widely known and are useful for understanding the intersections of categorised data - think of them as Venn diagrams on steroids. Let's go straight to an example:
</p>
<p>Say we have a load of information about dogs in the form of flags (<code>is_happy</code>, <code>is_yappy</code>, etc). Let's select out all of the data and aggregate:</p>
<pre><code class="lang-sql"><span class="hljs-keyword">SELECT</span>
is_happy,
is_yappy,
is_hairy,
is_waggy,
is_tall,
<span class="hljs-keyword">count</span>(*) <span class="hljs-keyword">AS</span> c
<span class="hljs-keyword">FROM</span> (
<span class="hljs-keyword">SELECT</span>
<span class="hljs-keyword">id</span>,
is_happy,
<span class="hljs-keyword">coalesce</span>(is_yappy, <span class="hljs-literal">FALSE</span>),
<span class="hljs-keyword">coalesce</span>(is_hairy, <span class="hljs-literal">FALSE</span>),
<span class="hljs-keyword">coalesce</span>(is_waggy, <span class="hljs-literal">FALSE</span>),
<span class="hljs-keyword">coalesce</span>(is_tall, <span class="hljs-literal">FALSE</span>)
<span class="hljs-keyword">FROM</span> happy_table
<span class="hljs-keyword">LEFT</span> <span class="hljs-keyword">JOIN</span> yappy_table <span class="hljs-keyword">USING</span> (<span class="hljs-keyword">id</span>)
...
)
<span class="hljs-keyword">GROUP</span> <span class="hljs-keyword">BY</span>
is_happy,
is_yappy,
is_hairy,
is_waggy,
is_tall
</code></pre>
<p>Note that the <code>count</code> in each row is a <b>distinct set of dogs</b> from the count in every other row.</p>
<pre><code class="lang-csv">is_happy is_yappy is_hairy is_waggy is_tall count
<span class="hljs-literal">TRUE</span> <span class="hljs-literal">FALSE</span> <span class="hljs-literal">TRUE</span> <span class="hljs-literal">FALSE</span> <span class="hljs-literal">TRUE</span> <span class="hljs-number">12</span>
<span class="hljs-literal">FALSE</span> <span class="hljs-literal">FALSE</span> <span class="hljs-literal">TRUE</span> <span class="hljs-literal">FALSE</span> <span class="hljs-literal">TRUE</span> <span class="hljs-number">7</span>
...
</code></pre>
<p>Now some setup:</p>
<pre><code class="lang-shell">pip <span class="hljs-keyword">install</span> upsetplot pandas
</code></pre>
<p>And the Python to make the plot:</p>
<pre><code class="lang-python"><span class="hljs-keyword">import</span> io
<span class="hljs-keyword">import</span> pandas <span class="hljs-keyword">as</span> pd
<span class="hljs-keyword">import</span> upsetplot
<span class="hljs-keyword">from</span> matplotlib <span class="hljs-keyword">import</span> pyplot <span class="hljs-keyword">as</span> plt
TSV = <span class="hljs-string">"""
is_happy is_yappy is_hairy is_waggy is_tall count
TRUE TRUE TRUE TRUE TRUE 12
TRUE TRUE TRUE TRUE FALSE 8
TRUE TRUE FALSE FALSE TRUE 23
TRUE FALSE TRUE TRUE TRUE 1
TRUE FALSE TRUE TRUE FALSE 3
FALSE FALSE TRUE FALSE TRUE 4
FALSE FALSE TRUE FALSE FALSE 5
"""</span>
df = pd.read_csv(io.StringIO(TSV), sep=<span class="hljs-string">"\t"</span>)
flags = [c <span class="hljs-keyword">for</span> c <span class="hljs-keyword">in</span> df.columns <span class="hljs-keyword">if</span> c != <span class="hljs-string">"count"</span>]
df = df.set_index(flags)
upsetplot.plot(df[<span class="hljs-string">"count"</span>])
plt.show()
</code></pre>
<p>The plot:</p>
<p><img src="images/upset.png" alt="UpSet Plot"></p>
<p>So, unhappy dogs are largely hairy and tall, there are a lot of tall yappy dogs, etc.</p>
<br>
<p><em><a href="upset-old.html">Old version</a> of this post munging data from a different format.</em></p>
</body>
</html>