nat
/
pride-data-analysis
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Finalize Milestone 2 modifications, in line with checklist

This commit is contained in:
Nat 2023-02-15 16:53:33 -08:00
parent 8b2b2e9e3f
commit 5c19c84bac
Signed by: nat
GPG Key ID: B53AB05285D710D6
5 changed files with 454 additions and 44 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
README.md
View File

@ -18,18 +18,19 @@ To quantify gaybourhoods, writers and data analysts from The Pudding collected i
## Team Members ## Team Members
- Nat Scott - Nat Scott: Student of computer and environmental science
- Sami Almuallim - Sami Almuallim: Student of computer and data science
## Images ## Images
{You should use this area to add a screenshot of an interesting plot, or of your dashboard} Images coming soon.
<img src ="images/test.png" width="100px">
## References ## References
{Add your stuff here} - [Men are from Chelsea, Women are from Park Slope](https://pudding.cool/2018/06/gayborhoods/)
- The article for which the data was originally collected.
- [The Gaybourhoods data set on Github](https://github.com/the-pudding/data/blob/master/gayborhoods/README.md)
Sources of (potential) secondary data sets:
- [Data set relating US ZIP codes to their coordinates](https://www.kaggle.com/datasets/joeleichter/us-zip-codes-with-lat-and-long)
- [Geographic situation of taxes payed in the US](https://www.irs.gov/statistics/soi-tax-stats-individual-income-tax-statistics-2018-zip-code-data-soi)

223
analysis/analysis1.ipynb
View File

@ -13,13 +13,13 @@
"source": [ "source": [
"## Research question/interests\n", "## Research question/interests\n",
"\n", "\n",
"Is there a correlation between political alignment & living in neighbourhoods with large quantities of LGBT people?\n", "**Is there a correlation between political alignment & living in neighbourhoods with large quantities of LGBT people?** The obvious answer to this question is \"yes, they are going to mostly be democrats\" but anyone who's ever been around queer people will know that this question is quite a bit more nuanced than that, and this nuance is what we hope to capture in investigating this question.\n",
"\n", "\n",
"- The gaybourhoods data set does not include data on residents political alignments, however, there is a wealth of electoral data available freely online that we intend on incorporating into this project. The primary difficulty then will be developing a geographic \"compatibility layer\" between the data sets so that the data can be understood in the same context. To build this, we intend on working with the OpenStreetMap API to create an additional column representing observations position space in a more neutral way, such as their coordinates.\n", "- The gaybourhoods data set does not include data on residents political alignments, however, there is a wealth of electoral data available freely online that we intend on incorporating into this project. The primary difficulty then will be developing a geographic \"compatibility layer\" between the data sets so that the data can be understood in the same context. To build this, we intend on working with the OpenStreetMap API to create an additional column representing observations position space in a more neutral way, such as their coordinates.\n",
"- Alternatively, we've also considered working with an additional data set that links US zip codes to their longitude and lattitude positions. As such, incorporating this data would be as easy as merging the two tables.\n", "- Alternatively, we've also considered working with an additional data set that links US zip codes to their longitude and lattitude positions. As such, incorporating this data would be as easy as merging the two tables.\n",
"\n", "\n",
"\n", "\n",
"Is there a correlation between geographical stratums & being LGBT?\n", "**Is there a correlation between geographical stratums & being LGBT?** This question is more abstract, and will serve as a preliminary exploration of the data in hopes of establishing two key details along the way that will shape the rest of the project: how do we quantify queerness, and how do we best represent it visually?\n",
"\n", "\n",
"- Once again, representing this data visually will require determining the coordinates associated with each observation.\n", "- Once again, representing this data visually will require determining the coordinates associated with each observation.\n",
"- The gaybourhoods data set defines a \"gaybourhood index\" which effectively measures how friendly a given neighbourhood is to queer people. Since this index is entirely subjective, we will need to closely evaluate it's usefulness for our project and investigate different ways to quantify \"queer-friendliness\"\n", "- The gaybourhoods data set defines a \"gaybourhood index\" which effectively measures how friendly a given neighbourhood is to queer people. Since this index is entirely subjective, we will need to closely evaluate it's usefulness for our project and investigate different ways to quantify \"queer-friendliness\"\n",
@ -27,6 +27,225 @@
"- Obviously, visualizing this among many aspects of the other research questions would involve projecting the data onto a map of the United States, so visualizing this research question would motivate many of the visualizations for other components of this project" "- Obviously, visualizing this among many aspects of the other research questions would involve projecting the data onto a map of the United States, so visualizing this research question would motivate many of the visualizations for other components of this project"
] ]
}, },
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>GEOID10</th>\n",
" <th>Tax_Mjoint</th>\n",
" <th>Mjoint_MF</th>\n",
" <th>Mjoint_SS</th>\n",
" <th>Mjoint_FF</th>\n",
" <th>Mjoint_MM</th>\n",
" <th>TaxRate_SS</th>\n",
" <th>TaxRate_FF</th>\n",
" <th>TaxRate_MM</th>\n",
" <th>Cns_TotHH</th>\n",
" <th>...</th>\n",
" <th>FF_Cns</th>\n",
" <th>FF_Index</th>\n",
" <th>MM_Tax</th>\n",
" <th>MM_Cns</th>\n",
" <th>MM_Index</th>\n",
" <th>SS_Index</th>\n",
" <th>SS_Index_Weight</th>\n",
" <th>Parade_Weight</th>\n",
" <th>Bars_Weight</th>\n",
" <th>TOTINDEX</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>90069</td>\n",
" <td>2120</td>\n",
" <td>1689</td>\n",
" <td>431</td>\n",
" <td>61</td>\n",
" <td>370</td>\n",
" <td>203.301887</td>\n",
" <td>28.773585</td>\n",
" <td>174.528302</td>\n",
" <td>12551</td>\n",
" <td>...</td>\n",
" <td>1.847099</td>\n",
" <td>6.724415</td>\n",
" <td>29.583721</td>\n",
" <td>18.704533</td>\n",
" <td>48.288254</td>\n",
" <td>55.012669</td>\n",
" <td>39.429995</td>\n",
" <td>10</td>\n",
" <td>17.647059</td>\n",
" <td>67.077054</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>94114</td>\n",
" <td>5080</td>\n",
" <td>4036</td>\n",
" <td>1044</td>\n",
" <td>170</td>\n",
" <td>874</td>\n",
" <td>205.511811</td>\n",
" <td>33.464567</td>\n",
" <td>172.047244</td>\n",
" <td>16456</td>\n",
" <td>...</td>\n",
" <td>4.161579</td>\n",
" <td>9.834048</td>\n",
" <td>29.163165</td>\n",
" <td>19.415304</td>\n",
" <td>48.578469</td>\n",
" <td>58.412517</td>\n",
" <td>41.866815</td>\n",
" <td>0</td>\n",
" <td>20.000000</td>\n",
" <td>61.866815</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>10011</td>\n",
" <td>5790</td>\n",
" <td>5166</td>\n",
" <td>624</td>\n",
" <td>97</td>\n",
" <td>527</td>\n",
" <td>107.772021</td>\n",
" <td>16.753022</td>\n",
" <td>91.018998</td>\n",
" <td>29762</td>\n",
" <td>...</td>\n",
" <td>1.531029</td>\n",
" <td>4.370779</td>\n",
" <td>15.428332</td>\n",
" <td>10.932081</td>\n",
" <td>26.360413</td>\n",
" <td>30.731192</td>\n",
" <td>22.026394</td>\n",
" <td>10</td>\n",
" <td>5.882353</td>\n",
" <td>37.908747</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>10014</td>\n",
" <td>3510</td>\n",
" <td>3229</td>\n",
" <td>281</td>\n",
" <td>74</td>\n",
" <td>207</td>\n",
" <td>80.056980</td>\n",
" <td>21.082621</td>\n",
" <td>58.974359</td>\n",
" <td>18786</td>\n",
" <td>...</td>\n",
" <td>2.482293</td>\n",
" <td>6.055939</td>\n",
" <td>9.996551</td>\n",
" <td>5.943318</td>\n",
" <td>15.939869</td>\n",
" <td>21.995808</td>\n",
" <td>15.765361</td>\n",
" <td>10</td>\n",
" <td>11.764706</td>\n",
" <td>37.530067</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>94103</td>\n",
" <td>2660</td>\n",
" <td>2417</td>\n",
" <td>243</td>\n",
" <td>34</td>\n",
" <td>209</td>\n",
" <td>91.353383</td>\n",
" <td>12.781955</td>\n",
" <td>78.571429</td>\n",
" <td>12728</td>\n",
" <td>...</td>\n",
" <td>0.837431</td>\n",
" <td>3.004058</td>\n",
" <td>13.318386</td>\n",
" <td>4.961779</td>\n",
" <td>18.280165</td>\n",
" <td>21.284224</td>\n",
" <td>15.255337</td>\n",
" <td>10</td>\n",
" <td>10.588235</td>\n",
" <td>35.843573</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>5 rows × 29 columns</p>\n",
"</div>"
],
"text/plain": [
" GEOID10 Tax_Mjoint Mjoint_MF Mjoint_SS Mjoint_FF Mjoint_MM \\\n",
"0 90069 2120 1689 431 61 370 \n",
"1 94114 5080 4036 1044 170 874 \n",
"2 10011 5790 5166 624 97 527 \n",
"3 10014 3510 3229 281 74 207 \n",
"4 94103 2660 2417 243 34 209 \n",
"\n",
" TaxRate_SS TaxRate_FF TaxRate_MM Cns_TotHH ... FF_Cns FF_Index \\\n",
"0 203.301887 28.773585 174.528302 12551 ... 1.847099 6.724415 \n",
"1 205.511811 33.464567 172.047244 16456 ... 4.161579 9.834048 \n",
"2 107.772021 16.753022 91.018998 29762 ... 1.531029 4.370779 \n",
"3 80.056980 21.082621 58.974359 18786 ... 2.482293 6.055939 \n",
"4 91.353383 12.781955 78.571429 12728 ... 0.837431 3.004058 \n",
"\n",
" MM_Tax MM_Cns MM_Index SS_Index SS_Index_Weight Parade_Weight \\\n",
"0 29.583721 18.704533 48.288254 55.012669 39.429995 10 \n",
"1 29.163165 19.415304 48.578469 58.412517 41.866815 0 \n",
"2 15.428332 10.932081 26.360413 30.731192 22.026394 10 \n",
"3 9.996551 5.943318 15.939869 21.995808 15.765361 10 \n",
"4 13.318386 4.961779 18.280165 21.284224 15.255337 10 \n",
"\n",
" Bars_Weight TOTINDEX \n",
"0 17.647059 67.077054 \n",
"1 20.000000 61.866815 \n",
"2 5.882353 37.908747 \n",
"3 11.764706 37.530067 \n",
"4 10.588235 35.843573 \n",
"\n",
"[5 rows x 29 columns]"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import pandas as pd\n",
"\n",
"gaybourhoods = pd.read_csv(\"../data/raw/gaybourhoods.csv\")\n",
"gaybourhoods.head(5)"
]
},
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": null, "execution_count": null,

226
analysis/analysis2.ipynb
View File

@ -13,16 +13,234 @@
"source": [ "source": [
"## Research question/interests\n", "## Research question/interests\n",
"\n", "\n",
"Is there a positive or a negative correlation between taxes paid and the pride of a given queer neighbourhood?\n", "**How are the different metrics of pride represented in this data set correlated?** Answering this question will provide a foundation upon which we can work to answer the more complicated questions that follow.\n",
"\n",
"- This will probably be the simplest research question, requiring only the data contained in our original data set. To explore this topic, we will use different visualization methods discussed in class to develop a better understanding of the data.\n",
"\n",
"**Is there a positive or a negative correlation between taxes paid and the pride of a given queer neighbourhood?** Taxes are influenced by a variety of socio-economic factors and we hope that in analyzing both tax data and our quantification of queerness on a geographic level, we'll be able to gleam insight into the question of how queerness and class are interrelated.\n",
"\n", "\n",
"- Similar again to the first research question posed, we'll need to find another data set containing geographically located tax data, which should be easy to acquire from the US government (for example, [in our cursory research, we found this data set from the IRS](https://www.irs.gov/statistics/soi-tax-stats-individual-income-tax-statistics-2018-zip-code-data-soi)).\n", "- Similar again to the first research question posed, we'll need to find another data set containing geographically located tax data, which should be easy to acquire from the US government (for example, [in our cursory research, we found this data set from the IRS](https://www.irs.gov/statistics/soi-tax-stats-individual-income-tax-statistics-2018-zip-code-data-soi)).\n",
"- This would bring the number of data sets used in this project up to three, which might present some challenges in terms of the amount of data wrangling necessary to bring it all together.\n", "- This would bring the number of data sets used in this project up to three, which might present some challenges in terms of the amount of data wrangling necessary to bring it all together.\n",
"- To measure this, we would rank the neighbourhoods presented in the gaybourhoods data set by pride (an open question which we will explore in a separate research question)\n", "- To measure this, we would rank the neighbourhoods presented in the gaybourhoods data set by pride (an open question which we will explore in a separate research question)"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n", "\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n", "\n",
"Is there a correlation between pride flags & parades & gay bars in a given region? In other words, how are the different metrics of pride represented in this data set correlated?\n", " .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>GEOID10</th>\n",
" <th>Tax_Mjoint</th>\n",
" <th>Mjoint_MF</th>\n",
" <th>Mjoint_SS</th>\n",
" <th>Mjoint_FF</th>\n",
" <th>Mjoint_MM</th>\n",
" <th>TaxRate_SS</th>\n",
" <th>TaxRate_FF</th>\n",
" <th>TaxRate_MM</th>\n",
" <th>Cns_TotHH</th>\n",
" <th>...</th>\n",
" <th>FF_Cns</th>\n",
" <th>FF_Index</th>\n",
" <th>MM_Tax</th>\n",
" <th>MM_Cns</th>\n",
" <th>MM_Index</th>\n",
" <th>SS_Index</th>\n",
" <th>SS_Index_Weight</th>\n",
" <th>Parade_Weight</th>\n",
" <th>Bars_Weight</th>\n",
" <th>TOTINDEX</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>90069</td>\n",
" <td>2120</td>\n",
" <td>1689</td>\n",
" <td>431</td>\n",
" <td>61</td>\n",
" <td>370</td>\n",
" <td>203.301887</td>\n",
" <td>28.773585</td>\n",
" <td>174.528302</td>\n",
" <td>12551</td>\n",
" <td>...</td>\n",
" <td>1.847099</td>\n",
" <td>6.724415</td>\n",
" <td>29.583721</td>\n",
" <td>18.704533</td>\n",
" <td>48.288254</td>\n",
" <td>55.012669</td>\n",
" <td>39.429995</td>\n",
" <td>10</td>\n",
" <td>17.647059</td>\n",
" <td>67.077054</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>94114</td>\n",
" <td>5080</td>\n",
" <td>4036</td>\n",
" <td>1044</td>\n",
" <td>170</td>\n",
" <td>874</td>\n",
" <td>205.511811</td>\n",
" <td>33.464567</td>\n",
" <td>172.047244</td>\n",
" <td>16456</td>\n",
" <td>...</td>\n",
" <td>4.161579</td>\n",
" <td>9.834048</td>\n",
" <td>29.163165</td>\n",
" <td>19.415304</td>\n",
" <td>48.578469</td>\n",
" <td>58.412517</td>\n",
" <td>41.866815</td>\n",
" <td>0</td>\n",
" <td>20.000000</td>\n",
" <td>61.866815</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>10011</td>\n",
" <td>5790</td>\n",
" <td>5166</td>\n",
" <td>624</td>\n",
" <td>97</td>\n",
" <td>527</td>\n",
" <td>107.772021</td>\n",
" <td>16.753022</td>\n",
" <td>91.018998</td>\n",
" <td>29762</td>\n",
" <td>...</td>\n",
" <td>1.531029</td>\n",
" <td>4.370779</td>\n",
" <td>15.428332</td>\n",
" <td>10.932081</td>\n",
" <td>26.360413</td>\n",
" <td>30.731192</td>\n",
" <td>22.026394</td>\n",
" <td>10</td>\n",
" <td>5.882353</td>\n",
" <td>37.908747</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>10014</td>\n",
" <td>3510</td>\n",
" <td>3229</td>\n",
" <td>281</td>\n",
" <td>74</td>\n",
" <td>207</td>\n",
" <td>80.056980</td>\n",
" <td>21.082621</td>\n",
" <td>58.974359</td>\n",
" <td>18786</td>\n",
" <td>...</td>\n",
" <td>2.482293</td>\n",
" <td>6.055939</td>\n",
" <td>9.996551</td>\n",
" <td>5.943318</td>\n",
" <td>15.939869</td>\n",
" <td>21.995808</td>\n",
" <td>15.765361</td>\n",
" <td>10</td>\n",
" <td>11.764706</td>\n",
" <td>37.530067</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>94103</td>\n",
" <td>2660</td>\n",
" <td>2417</td>\n",
" <td>243</td>\n",
" <td>34</td>\n",
" <td>209</td>\n",
" <td>91.353383</td>\n",
" <td>12.781955</td>\n",
" <td>78.571429</td>\n",
" <td>12728</td>\n",
" <td>...</td>\n",
" <td>0.837431</td>\n",
" <td>3.004058</td>\n",
" <td>13.318386</td>\n",
" <td>4.961779</td>\n",
" <td>18.280165</td>\n",
" <td>21.284224</td>\n",
" <td>15.255337</td>\n",
" <td>10</td>\n",
" <td>10.588235</td>\n",
" <td>35.843573</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>5 rows × 29 columns</p>\n",
"</div>"
],
"text/plain": [
" GEOID10 Tax_Mjoint Mjoint_MF Mjoint_SS Mjoint_FF Mjoint_MM \\\n",
"0 90069 2120 1689 431 61 370 \n",
"1 94114 5080 4036 1044 170 874 \n",
"2 10011 5790 5166 624 97 527 \n",
"3 10014 3510 3229 281 74 207 \n",
"4 94103 2660 2417 243 34 209 \n",
"\n", "\n",
"- This will probably be the simplest research question, requiring only the data contained in our original data set. To explore this topic, we will use different visualization methods discussed in class to develop a better understanding of the data." " TaxRate_SS TaxRate_FF TaxRate_MM Cns_TotHH ... FF_Cns FF_Index \\\n",
"0 203.301887 28.773585 174.528302 12551 ... 1.847099 6.724415 \n",
"1 205.511811 33.464567 172.047244 16456 ... 4.161579 9.834048 \n",
"2 107.772021 16.753022 91.018998 29762 ... 1.531029 4.370779 \n",
"3 80.056980 21.082621 58.974359 18786 ... 2.482293 6.055939 \n",
"4 91.353383 12.781955 78.571429 12728 ... 0.837431 3.004058 \n",
"\n",
" MM_Tax MM_Cns MM_Index SS_Index SS_Index_Weight Parade_Weight \\\n",
"0 29.583721 18.704533 48.288254 55.012669 39.429995 10 \n",
"1 29.163165 19.415304 48.578469 58.412517 41.866815 0 \n",
"2 15.428332 10.932081 26.360413 30.731192 22.026394 10 \n",
"3 9.996551 5.943318 15.939869 21.995808 15.765361 10 \n",
"4 13.318386 4.961779 18.280165 21.284224 15.255337 10 \n",
"\n",
" Bars_Weight TOTINDEX \n",
"0 17.647059 67.077054 \n",
"1 20.000000 61.866815 \n",
"2 5.882353 37.908747 \n",
"3 11.764706 37.530067 \n",
"4 10.588235 35.843573 \n",
"\n",
"[5 rows x 29 columns]"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import pandas as pd\n",
"\n",
"gaybourhoods = pd.read_csv(\"../data/raw/gaybourhoods.csv\")\n",
"gaybourhoods.head(5)"
] ]
}, },
{ {

28
analysis/analysis3.ipynb
View File

@ -1,28 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"source": [
"# Your Name Here"
],
"metadata": {}
},
{
"cell_type": "markdown",
"source": [
"## Research question/interests\r\n",
"\r\n",
"Briefly describe your research question or interests here."
],
"metadata": {}
}
],
"metadata": {
"orig_nbformat": 4,
"language_info": {
"name": "plaintext"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

0
data/raw/data.csv → data/raw/gaybourhoods.csv
View File