In this project find datasets on the web and visualize these datasets.
Although it is extremely difficult to quantify human psychology and feelings, we will use data on happiness and charity as reference indicators. Add factor data to your dataset and explore relationships between each piece of data.
The dataset is based on country names. As there may be discrepancies in names between reference source data, we have unified them to the extent possible. Please note that not all countries in the world are included.
Regions and regional categories covered by the dataset
Questions about domain categories and regions of interest
import numpy as np
import pandas as pd
import seaborn as sns
import scipy.stats as stats
import plotly.express as px
import matplotlib.pyplot as plt
from matplotlib.cm import get_cmap
from matplotlib.patheffects import withStroke
import warnings
warnings.simplefilter('ignore', FutureWarning)
import plotly.express and arrange DataFrame to plot ranking
url_G20 = 'https://en.wikipedia.org/wiki/G20'
url_WGI = 'https://en.wikipedia.org/wiki/World_Giving_Index'
url_WHR = 'https://en.wikipedia.org/wiki/World_Happiness_Report'
Extract country names from G20 Member without African Unioin and European Union
g20 = pd.read_html(url_G20)[3]
g20 = g20['Member'][:19]
g20
0 Argentina 1 Australia 2 Brazil 3 Canada 4 China 5 France 6 Germany 7 India 8 Indonesia 9 Italy 10 Japan 11 Mexico 12 South Korea 13 Russia 14 Saudi Arabia 15 South Africa 16 Turkey 17 United Kingdom 18 United States Name: Member, dtype: object
Create giving data frame
Nan values are replaced by the average value for each country for later normalization.
df = pd.read_html(url_WGI)[1]
df.columns = df.columns.str.replace(r'(?<=\d{4}).*', '', regex=True)
Fill Nan values
giv = df.set_index('Country')
name_mapping = {'Macedonia':'North Macedonia'}
giv = giv.rename(index=name_mapping)
giving = giv.T.fillna(giv.T.mean(axis=0), axis=0).T
giving.sample(5)
| 2022 | 2021 | 2019 | 2018 | 2017 | 2016 | 2015 | 2014 | 2013 | |
|---|---|---|---|---|---|---|---|---|---|
| Country | |||||||||
| Spain | 99.0 | 100.0 | 58.0 | 54.0 | 71.0 | 79.0 | 58.0 | 62.0 | 57.0 |
| Zambia | 9.0 | 17.0 | 32.0 | 21.0 | 18.0 | 63.0 | 51.0 | 47.0 | 45.0 |
| France | 100.0 | 106.0 | 66.0 | 72.0 | 67.0 | 81.0 | 74.0 | 90.0 | 77.0 |
| Argentina | 21.0 | 74.0 | 75.0 | 94.0 | 83.0 | 83.0 | 108.0 | 77.0 | 78.0 |
| Serbia | 27.0 | 48.0 | 123.0 | 129.0 | 132.0 | 135.0 | 118.0 | 124.0 | 123.0 |
Merge World_Giving_Index dataframe and G20 country name
Replace rank[number] strings
Normalize rank data
df = giving[giving.columns[::-1]]
df = df.loc[g20]
give_20 = 1 - (df - df.min())/(df.max() - df.min())
give_20.head()
| 2013 | 2014 | 2015 | 2016 | 2017 | 2018 | 2019 | 2021 | 2022 | |
|---|---|---|---|---|---|---|---|---|---|
| Country | |||||||||
| Argentina | 0.416667 | 0.401575 | 0.251748 | 0.413043 | 0.404412 | 0.340426 | 0.408 | 0.353982 | 0.829060 |
| Australia | 0.954545 | 0.960630 | 0.972028 | 0.992754 | 0.970588 | 0.992908 | 0.976 | 0.964602 | 0.974359 |
| Brazil | 0.583333 | 0.566929 | 0.580420 | 0.608696 | 0.602941 | 0.489362 | 0.416 | 0.530973 | 0.854701 |
| Canada | 0.992424 | 0.984252 | 0.979021 | 0.971014 | 0.963235 | 0.907801 | 0.960 | 0.699115 | 0.940171 |
| China | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000 | 0.168142 | 0.589744 |
sort dataframe rank and year,
to make default 2013 rank
in these case need to add kind option
t_ = give_20.stack().reset_index()
t_.columns = ['country', 'year', 'rank']
t_rank = t_.sort_values('rank', ascending=False).sort_values('year',ascending=True,kind='stable')
t_rank.head()
| country | year | rank | |
|---|---|---|---|
| 162 | United States | 2013 | 1.000000 |
| 27 | Canada | 2013 | 0.992424 |
| 153 | United Kingdom | 2013 | 0.962121 |
| 9 | Australia | 2013 | 0.954545 |
| 72 | Indonesia | 2013 | 0.878788 |
fig = px.bar(t_rank,
x = "rank",
y = "country",
color = 'country',
animation_frame = "year",
title="G20 Countries Charitable Trend from 2013 to 2022"
)
fig.update_layout(margin=dict(l=0, r=0, t=30, b=0))
fig.layout.updatemenus[0].buttons[0].args[1]["frame"]["duration"] = 800
fig.show()
In the case below, it use receplotly to move the y-axis, so the rank order will always preserved.
from raceplotly.plots import barplot
my_raceplot = barplot(t_,
item_column='country',
value_column='rank',
time_column='year',
top_entries = 20,
)
my_raceplot.plot(title = 'Changes in the Giving index Ranks of G20 countries from 2013 to 2022',
item_label = 'Top 20 Giveng Countries',
value_label = 'Rank',
frame_duration = 1000)
Show rank in line chart
fig = px.line(t_,
x = "year",
y = "rank",
color = 'country',
markers = True
)
fig.show()
Load Data from World Happiness Report extract 2023 report
There is data for only 20 countries in 2022, and no data in 2021.
The 2023 report includes average happiness scores from 2020 to 2022.
Hence it will use data from 2023.
df = pd.read_html(url_WHR)[12:-1]
Make dataFrame from each table
c_2023 = df[0]
# c_2022 = df[2]
c_2020 = df[4]
c_2019 = df[6]
c_2018 = df[8]
c_2017 = df[10]
c_2016 = df[13]
c_2013 = df[15]
Wikipedia has a useful collection of information, but there are some years where data is missing.
Data for 2022, 2021, and 2015 are obtained from the World Happiness Report web page.
There is also 2012, add it.
y_2022 = 'Appendix_2_Data_for_Figure_2.1.xls'
y_2021 = 'DataForFigure2.1WHR2021C2.xls'
y_2015 = 'Chapter2OnlineData_Expanded-with-Trust-and-Governance.xlsx'
y_2012 = '2012.xlsx'
c_2022 = pd.read_excel(y_2022)
c_2022 = c_2022.iloc[:,[0,1]]
c_2021 = pd.read_excel(y_2021)
c_2021['2021'] = range(1, c_2021.shape[0] +1)
c_2021 = c_2021.iloc[:,[-1,0]]
ex = pd.ExcelFile(y_2015)
c_2015 = ex.parse(ex.sheet_names[1])
c_2015.columns = c_2015.iloc[2]
c_2015 = c_2015.iloc[3:,[1]].reset_index(drop=True)
exl = pd.ExcelFile('2012.xlsx')
c_2012 = exl.parse(exl.sheet_names[1])
c_2012 = c_2012.iloc[:,[0]]
Edit dataframe
c_2023.columns = ['2023','country']
c_2023 = c_2023.iloc[:,[1,0]]
c_2022.columns = ['2022','country']
c_2021.columns = ['2021','country']
c_2020 = c_2020.iloc[:,:2]
c_2020.columns = ['2020','country']
c_2019 = c_2019.iloc[:,:2]
c_2019.columns = ['2019','country']
c_2018 = c_2018.iloc[:,:2]
c_2018.columns = ['2018','country']
c_2017 = c_2017.iloc[:,[0,2]]
c_2017.columns = ['2017','country']
c_2016 = c_2016.iloc[:,:2]
c_2016.columns = ['2016','country']
c_2015['2015'] = range(1, c_2015.shape[0] +1)
c_2015 = c_2015.iloc[:,[1,0]]
c_2013 = c_2013.iloc[:,:2]
c_2013.columns = ['2013','country']
c_2012['2012'] = range(1, c_2012.shape[0] +1)
c_2012.columns = ['country', '2012']
c_2012 = c_2012.iloc[:,[1,0]]
Replace and unify the names so that they have the same country name
def replace_strings_in_dataframe(df, replacements, col='country'):
df.loc[:,col] = df[col].replace(replacements, regex=True)
return df
replacements = {
'&': 'and',
r'(?<=\w)\*': '',
'region': 'Region',
r'(?<=\w)\[\w+?\]': '',
'Czechia':'Czech Republic',
'Slovak Republic': 'Slovakia',
'Trkiye': 'Turkey',
'Burma':'Myanmar',
'Viet.+$':'Vietnam',
'The Gambia': 'Gambia',
'Kyrgyz Republic': 'Kyrgyzstan',
r'^Korea': 'South Korea',
'Lao P.D.R.': 'Laos',
"Cote.+$|Côte.+$": "Ivory Coast",
'North Cyprus':'Northern Cyprus',
r'^Macedonia$': 'North Macedonia',
'Eswatini.+$|Swaziland': 'Eswatini',
'Taiwan.+$': 'Taiwan',
'Maca.+$|Maca':'Macao',
r'^Somaliland$': 'Somaliland Region',
"People's Republic of China": 'China',
'Hong Kong.+$': 'Hong Kong',
'United States.+$|USA': 'United States',
r'Palestin.+$|West Bank':'Palestinian Territory',
r'Congo, Rep.+$|^Republic of (the )?Congo|Congo \(Brazzaville\)': 'Congo',
r'Congo, Democratic Republic of|Democratic Congo|Congo \(Kinshasa\)|DR Congo': 'Democratic Republic of the Congo',
}
df_name_list = [c_2023,c_2022,c_2021,c_2020,c_2019,c_2018,c_2017,c_2016,c_2015,c_2013,c_2012]
for i in df_name_list:
replace_strings_in_dataframe(i, replacements)
Merge each years data frame
Define merge function
The Nan value will be out of rank and will be replaced by the max value here for normalization later.
def merge_dataframes(dfs, on_key='country'):
merged_df = dfs[0]
for df in dfs[1:]:
merged_df = merged_df.merge(df, on=on_key, how='outer')
return merged_df
df = merge_dataframes(df_name_list)
df = df[df.columns[::-1]].set_index('country')
df = df.apply(pd.to_numeric, errors='coerce')
# happiness = happiness.fillna(happiness.max())
happiness = df.T.fillna(df.T.mean(axis=0), axis=0).T
indexes_to_drop = ['World', 'Europe','xx']
happiness = happiness.drop(indexes_to_drop)
happiness.tail(5)
| 2012 | 2013 | 2015 | 2016 | 2017 | 2018 | 2019 | 2020 | 2021 | 2022 | 2023 | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| country | |||||||||||
| Somaliland Region | 111.0 | 99.666667 | 91.0 | 97.0 | 99.666667 | 99.666667 | 99.666667 | 99.666667 | 99.666667 | 99.666667 | 99.666667 |
| Oman | 22.5 | 23.000000 | 22.0 | 22.5 | 22.500000 | 22.500000 | 22.500000 | 22.500000 | 22.500000 | 22.500000 | 22.500000 |
| Djibouti | 95.0 | 110.500000 | 126.0 | 110.5 | 110.500000 | 110.500000 | 110.500000 | 110.500000 | 110.500000 | 110.500000 | 110.500000 |
| Guyana | 45.0 | 45.000000 | 45.0 | 45.0 | 45.000000 | 45.000000 | 45.000000 | 45.000000 | 45.000000 | 45.000000 | 45.000000 |
| Cuba | 69.0 | 69.000000 | 69.0 | 69.0 | 69.000000 | 69.000000 | 69.000000 | 69.000000 | 69.000000 | 69.000000 | 69.000000 |
Select G20 countries and normalize their ranks.
df = happiness.loc[g20]
happy_20 = 1 - (df - df.min())/(df.max() - df.min())
happy_20.head()
| 2012 | 2013 | 2015 | 2016 | 2017 | 2018 | 2019 | 2020 | 2021 | 2022 | 2023 | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| country | |||||||||||
| Argentina | 0.682243 | 0.809760 | 0.776786 | 0.821429 | 0.852174 | 0.825397 | 0.709924 | 0.669173 | 0.640625 | 0.637097 | 0.649123 |
| Australia | 0.962617 | 0.966915 | 0.955357 | 0.973214 | 0.982609 | 0.976190 | 0.984733 | 0.992481 | 1.000000 | 1.000000 | 1.000000 |
| Brazil | 0.813084 | 0.851117 | 0.901786 | 0.901786 | 0.869565 | 0.833333 | 0.824427 | 0.842105 | 0.812500 | 0.790323 | 0.675439 |
| Canada | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 0.976562 | 0.975806 | 0.991228 |
| China | 0.000000 | 0.345740 | 0.294643 | 0.312500 | 0.373913 | 0.373016 | 0.358779 | 0.375940 | 0.429688 | 0.516129 | 0.543860 |
h_ = happy_20.stack().reset_index()
h_.columns = ['country', 'year', 'rank']
h_rank = h_.sort_values('rank', ascending=False).sort_values('year',ascending=True,kind='stable')
h_rank.head()
| country | year | rank | |
|---|---|---|---|
| 33 | Canada | 2012 | 1.000000 |
| 11 | Australia | 2012 | 0.962617 |
| 198 | United States | 2012 | 0.943925 |
| 187 | United Kingdom | 2012 | 0.878505 |
| 55 | France | 2012 | 0.831776 |
fig = px.bar(h_rank,
x = "rank",
y = "country",
color = 'country',
animation_frame = "year",
title="G20 Countries Happiness Trend from 2012 to 2023"
)
fig.update_layout(margin=dict(l=0, r=0, t=50, b=0))
fig.layout.updatemenus[0].buttons[0].args[1]["frame"]["duration"] = 800
fig.show()
giving_2022 = giving.iloc[:,:1].reset_index()
giving_2022.columns = ['country','giving']
giving_2022
| country | giving | |
|---|---|---|
| 0 | Indonesia | 1.000000 |
| 1 | Kenya | 2.000000 |
| 2 | United States | 3.000000 |
| 3 | Australia | 4.000000 |
| 4 | New Zealand | 5.000000 |
| ... | ... | ... |
| 154 | Suriname | 66.000000 |
| 155 | Syria | 43.666667 |
| 156 | Trinidad and Tobago | 16.500000 |
| 157 | Turkmenistan | 38.000000 |
| 158 | Yemen | 136.000000 |
159 rows × 2 columns
Extract 2023 form happiness rank dataframe
Merge two dataframes.
Specify outer to avoid deleting Nan when merge two dataframes.
normalize each rank value
calculate socores
happy = happiness[['2023']].reset_index()
happy.columns = ['country','happiness']
df = happy.merge(giving_2022, on='country', how='outer')
Average values cannot be obtained because no past data exists. Therefore, replace it with the mean value.
df.giving = df.giving.fillna(df.giving.mean())
df_h = df['happiness']
df['norm_happiness'] = 1 - (df_h - df_h.min()) / (df_h.max() - df_h.min())
df_g = df['giving']
df['norm_giving'] = 1 - (df_g - df_g.min()) / ( df_g.max() - df_g.min())
df['diff'] = 1- abs(df['norm_happiness'] - df['norm_giving'])
df['ttl'] = df['norm_happiness'] + df['norm_giving']
df_give_happy = df.set_index('country')
df_give_happy = df_give_happy.apply(pd.to_numeric, errors='coerce')
df_give_happy['happiness'] = df_give_happy['happiness'].astype('int')
df_give_happy['giving'] = df_give_happy['giving'].astype('int')
sns.set()
sort_by = 'norm_giving'
df= df_give_happy.sort_values(sort_by, ascending=False)
df = df.iloc[:50]
hap = 'Happiness ranking'
giv = 'Giving ranking'
bet = 'Gap between happiness and charity'
x = df.index
y_hap = df['norm_happiness']
y_giv = df['norm_giving']
fig, ax = plt.subplots(figsize=(18, 7))
ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)
ax.spines['left'].set_visible(False)
ax.set_yticks([])
shadow = withStroke(linewidth=4, foreground='w')
plt.bar(x, y_giv, color='gray', alpha=.5, label='with Dark Color: G20 Member')
plt.bar(x, y_giv, color='#f5e66b', alpha=.9, width = 0.8, label=giv)
for g in g20:
if g in x:
plt.bar(g, df[df.index==g].norm_giving, color='gray', alpha=0.4)
plt.vlines(x, y_giv, y_hap, color='#f482bd', alpha = 1, label=bet)
plt.scatter(x, y_hap, color='#f482bd', alpha = 1, s = 300, label=hap, zorder=3).set_path_effects([shadow])
for i in range(len(x)):
plt.annotate("{}".format(df['happiness'][i]),xy=(x[i], y_hap[i]),
ha="center", va="center", color='w', fontsize=12)
plt.xlabel("Charitable Countries", fontsize = 14)
plt.ylabel("Charitable Ranking", fontsize = 14)
plt.title("Most Charitable Countries Rank with Happiness Rank Top 50", fontsize = 20)
plt.legend(shadow = True, fontsize = 12, loc = 'lower right', bbox_to_anchor=(1, .0))
plt.xticks(fontsize = 12)
fig.autofmt_xdate(rotation=45)
plt.show()
The graph shows that the countries with the highest philanthropy rankings do not necessarily have the highest levels of happiness.
The gray bar shows that countries other than the G20 are also passionate about charity.
It is also possible that charity demand may temporarily increase in the event of a natural disaster.
Change the sorting method and plot a graph in order of happiness level
sort_by = 'norm_happiness'
df = df_give_happy.sort_values(sort_by, ascending=False)
df = df.iloc[:50]
df.norm_giving = df.norm_giving.fillna(0)
hap = 'Happiness ranking'
giv = 'Giving ranking'
bet = 'Gap between happiness and charity'
x = df.index
y_hap = df['norm_happiness']
y_giv = df['norm_giving']
fig, ax = plt.subplots(figsize=(18, 7))
plt.bar(x, y_giv, color='gray', alpha=.5, label='with Dark Color: G20 Member')
plt.bar(x, y_giv, color='#f5e66b', alpha=.9, width = 0.8, label=giv)
for g in g20:
if g in x:
plt.bar(g, df[df.index==g].norm_giving, color='gray', alpha=0.4)
plt.vlines(x, y_giv, y_hap, color='#f482bd', label=bet)
plt.scatter(x, y_hap, color='#f482bd', s = 300, label=hap, zorder=3).set_path_effects([shadow])
for i in range(len(x)):
plt.annotate("{}".format(df['happiness'][i]),xy=(x[i], y_hap[i]),
ha="center", va="center", color='w', fontsize=14)
plt.xlabel("Happiness Countries", fontsize=14)
plt.ylabel("World Happiness Rank", fontsize=14)
plt.title("Most Happiness Rank with Charitiable Rank Top 50", fontsize=18)
plt.legend(shadow=True, fontsize=12, loc='upper right', bbox_to_anchor=(1,1))
plt.xticks(fontsize=12)
fig.autofmt_xdate(rotation=45)
plt.show()
It's the same as the previous graph, just looking at it differently.
The smaller the difference, that is, the shorter the pink line, the stronger the correlation between happiness and charity.
Let's take a look.
To plot this, sort by the diff column added during normalization.
sort_by = 'diff'
df= df_give_happy.sort_values(sort_by, ascending=False)
df = df.iloc[:50]
diff = 'Happiness Rank and Charitiable Rank Correlative Country'
diff_color = 'Countries (with Color : G20 Member)'
x = df.index
y_diff = df['diff']
fig, ax = plt.subplots(figsize=(18, 5))
plt.bar(x, y_diff, color='silver', alpha=0.5, width = 0.8, label=diff_color)
ax_1 = df.plot(kind='bar', label = diff)
plt.close(ax_1.figure)
for ticks in ax_1.xaxis.get_major_ticks():
if ticks.label1.get_text() in list(g20):
ax.patches[df.index.get_indexer([ticks.label1.get_text()])[0]].set_facecolor('deepskyblue')
plt.xlabel("Correlation Countries", fontsize=14)
plt.ylabel("Correlation Rank", fontsize=14)
plt.title("Most Correlative Countries Top 50", fontsize=18)
plt.legend(shadow=True, fontsize=12, loc='lower right', bbox_to_anchor=(.98, 0.05))
fig.autofmt_xdate(rotation=45)
plt.show()
In the UK, happiness and charity seem to be equally correlated.
Chad, Hong Kong, and Congo also seem to have a strong correlation.
Next, let's take a look at the top countries in descending order of their combined happiness and charity scores.
sort_by = 'ttl'
df= df_give_happy.sort_values(sort_by, ascending=False)
df = df.iloc[:50]
y_diff = df['ttl']
x = df.index
top = 'Happiness Score'
bot = 'Charities Score'
wit_col = 'with dark color : G20 Member'
fig, ax = plt.subplots(figsize=(18, 6))
plt.bar(x, df['ttl'], label = wit_col, color= 'w', alpha=0.3, width = 0.8)
df_tll = df[['ttl']]
ax_1 = df_tll.plot(kind='bar', alpha=0.5, width = 0.8)
plt.close(ax_1.figure)
for ticks in ax_1.xaxis.get_major_ticks():
if ticks.label1.get_text() in list(g20):
ax.patches[df.index.get_indexer([ticks.label1.get_text()])[0]].set_facecolor('gray')
plt.bar(x, df['norm_happiness'], label = top, color= 'goldenrod',alpha=.4, width = 0.8)
plt.bar(x, df['norm_giving'], label = bot, color= 'lightgreen', alpha=.6, width = 0.8,
bottom = df['norm_happiness'])
plt.xlabel("Total Rank", fontsize=14)
plt.ylabel("Total Score", fontsize=14)
plt.title("Happiness and Charitiable Total Score Rank Top 50", fontsize=18)
handles, labels = plt.gca().get_legend_handles_labels()
order = [0, 2, 1]
plt.legend([handles[idx] for idx in order], [labels[idx] for idx in order],shadow=True, fontsize=12)
fig.autofmt_xdate(rotation=45)
plt.show()
Oceania and North America rank highest.
Most of the top countries are in Europe. In Asia, Singapore is the highest, ranking 15th.
Many of the top countries are characterized by a nearly 1:1 ratio between happiness and charity.
What I would like to focus on here are countries such as Indonesia and Kenya, which do not have a high percentage of happiness, but are ranked because of their high level of charity. These countries have high scores overall.
A question arises here. Are countries with higher GDP happier and more charitable?
Now, let's import GDP data.
url = 'https://en.wikipedia.org/wiki/List_of_countries_by_GDP_(nominal)'
df = pd.read_html(url)[2]
df.columns = df.columns.droplevel()
After merging, the data for Northern Cyprus and Somaliland will missing, so we will add them as new data.
We use the IMF data as the standard, but since it contains missing values, we replace it with the average value between the World Bank and the United Nations.
gd = df.iloc[1:,[0,2,4,6]]
gd.columns = ['country','IMF','WB','UN']
gd = replace_strings_in_dataframe(gd, replacements)
def missing_country_data(df):
a = df.country.values
b = df_give_happy.index.values
return [i for i in b if i not in a]
missing_country_data(gd)
['Northern Cyprus', 'Somaliland Region']
new_data = {'country': ['Northern Cyprus', 'Somaliland Region'],
'additoinal': [4234, 2500],}
new = pd.DataFrame(new_data)
gd = pd.concat([gd,new], ignore_index=True)
gd = gd.set_index('country').apply(pd.to_numeric, errors='coerce')
gd = gd.T.fillna(gd.T.mean(axis=0), axis=0).T
gd = gd.sort_values('IMF', ascending=False)
pd.options.mode.chained_assignment = None # default='warn'
gd['gdp'] = range(1, gd.shape[0] +1)
gdp = gd.iloc[:,-1].reset_index()
df = df_give_happy.merge(gdp, on='country',how='left')
df_h = df['gdp']
df['norm_gdp'] = 1 - (df_h - df_h.min()) / (df_h.max() - df_h.min())
gdp.sample(5)
df_t = df['ttl']
df['norm_ttl']= (df_t - df_t.min()) / (df_t.max() - df_t.min())
df_g = df['norm_giving']
df_h = df['norm_happiness']
df['norm_g2'] = df_g * (df['norm_ttl'] / (df_g + df_h))
df['norm_h2'] = df_h * (df['norm_ttl'] / (df_g + df_h))
df_give_happy_gdp = df.copy()
df_give_happy_gdp
| country | happiness | giving | norm_happiness | norm_giving | diff | ttl | gdp | norm_gdp | norm_ttl | norm_g2 | norm_h2 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Finland | 1 | 74 | 1.000000 | 0.493056 | 0.493056 | 1.493056 | 48 | 0.760204 | 0.778587 | 0.257115 | 0.521472 |
| 1 | Denmark | 2 | 24 | 0.993407 | 0.840278 | 0.846871 | 1.833684 | 38 | 0.811224 | 0.958232 | 0.439106 | 0.519126 |
| 2 | Iceland | 3 | 23 | 0.986813 | 0.847222 | 0.860409 | 1.834035 | 113 | 0.428571 | 0.958417 | 0.442735 | 0.515682 |
| 3 | Israel | 4 | 78 | 0.980220 | 0.465278 | 0.485058 | 1.445498 | 29 | 0.857143 | 0.753505 | 0.242539 | 0.510967 |
| 4 | Netherlands | 5 | 29 | 0.973626 | 0.805556 | 0.831929 | 1.779182 | 17 | 0.918367 | 0.929488 | 0.420842 | 0.508646 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 162 | Somaliland Region | 99 | 145 | 0.349451 | 0.000000 | 0.650549 | 0.349451 | 184 | 0.066327 | 0.175459 | 0.000000 | 0.175459 |
| 163 | Oman | 22 | 145 | 0.858242 | 0.000000 | 0.141758 | 0.858242 | 69 | 0.653061 | 0.443792 | 0.000000 | 0.443792 |
| 164 | Djibouti | 110 | 145 | 0.278022 | 0.000000 | 0.721978 | 0.278022 | 171 | 0.132653 | 0.137788 | 0.000000 | 0.137788 |
| 165 | Guyana | 45 | 145 | 0.709890 | 0.000000 | 0.290110 | 0.709890 | 136 | 0.311224 | 0.365552 | 0.000000 | 0.365552 |
| 166 | Cuba | 69 | 145 | 0.551648 | 0.000000 | 0.448352 | 0.551648 | 68 | 0.658163 | 0.282097 | 0.000000 | 0.282097 |
167 rows × 12 columns
Outputs a color bar to visually confirm the colors to use for the bar plot.
cmap = plt.get_cmap('spring')
gradient = np.linspace(0, 1, 256).reshape(1, -1)
gradient = np.vstack((gradient, gradient))
fig, ax = plt.subplots(figsize=(20, 1))
ax.set_title('Spring Colormap')
ax.set_xticks([])
ax.set_yticks([])
ax.imshow(gradient, aspect='auto', cmap=cmap)
for i in np.linspace(0, 1, 11):
position = int(i * 255)
plt.text(position, 2, f'{i:.1f}', color='black', fontsize=12, ha='center')
plt.show()
def plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title, begin=0, end=50):
df = df.sort_values(sort_by, ascending=False).set_index('country')
df = df.iloc[begin:end]
cmap = plt.get_cmap('spring')
col_g = cmap(0.9) #'yellow'
col_h = cmap(0.35) #'pink'
top = 'Happiness Score'
bot = 'Charities Score'
withg20 = 'with dark color : G20 Member'
shadow = withStroke(linewidth=4, foreground='w')
x = df.index
df_tll = df[['norm_ttl']]
fig, ax = plt.subplots(figsize=(18, 6))
plt.bar(x, df['norm_ttl'], color='gray', alpha=.3, label='with Dark Color: G20 Member')
plt.bar(x, df['norm_ttl'], label = '', color= 'w', alpha=1, width = 0.8)
for g in g20:
if g in x:
plt.bar(g, df[df.index==g].norm_ttl, color='gray', alpha=0.4)
plt.bar(x, df['norm_h2'], label = top, color = col_h, alpha=.6, width = 0.8)
plt.bar(x, df['norm_g2'], label = bot, color = col_g, alpha=.6, width = 0.8,
bottom = df['norm_h2'])
avg_norm_ttl = df.norm_ttl.mean()
plt.axhline(avg_norm_ttl, color='lightskyblue', linestyle='--', label='Average')
plt.text(len(df) + 1, avg_norm_ttl,
f'Avg: {avg_norm_ttl:.2f}',
va='top', ha='right', color='b').set_path_effects([shadow])
x_position = len(df) / 2 * 1.1
x_adjust = x_position * 0.2
y_positoin = df['norm_ttl'].max()
y_adjust = y_positoin * 0.0
plt.text(x_position - x_adjust, y_positoin- y_adjust,
f'Avg Charities: {df.norm_g2.mean():.2f}',
va = 'bottom',
ha = 'right',
color = cmap(0.73),
fontsize = 12
).set_path_effects([shadow])
text_obj = plt.text(x_position + x_adjust, y_positoin-y_adjust,
f'Avg Happiness: {df.norm_h2.mean():.2f}',
va = 'bottom',
ha = 'right',
color = cmap(.3),
fontsize = 12
).set_path_effects([shadow])
plt.xlabel(xlabel, fontsize=14)
plt.ylabel('Total Score', fontsize=14)
plt.title(title, fontsize=20, pad=18)
handles, labels = plt.gca().get_legend_handles_labels()
order = [0, 1, 3, 2]
plt.legend([handles[idx] for idx in order],
[labels[idx] for idx in order],
shadow=True, fontsize=12,
loc='upper right')
fig.autofmt_xdate(rotation=45)
return plt.show()
df = df_give_happy_gdp
sort_by = 'norm_gdp'
xlabel = 'GDP Rank'
title = 'Happiness and Charitiable Total Score Rank Top 50 sort by GDP'
plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title)
Since the countries are ranked by GDP, the top countries are in the G20.
What I would like to note here is that the height of the bar is not constant.
The prediction that economically large countries are happy and benevolent is overturned.
The question of whether economic powers are happy or charitable is that they are not proportionately.
For example, Japan, which is ranked 3rd, has a bar for happiness but no charity, and its total score is about the same as Iraq, which ranks 50th.
Bangladesh and Egypt have very low scores.
The average score of the top 50 countries is 0.7.
So what happens below the top 50?
df = df_give_happy_gdp
sort_by = 'norm_gdp'
xlabel = 'GDP Rank'
title = 'Happiness and Charitiable Total Score Rank 51 to 100 sort by GDP'
plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title, begin=50, end =100)
New Zealand and Iceland stand out in this graph.
The average score for the top 50 countries was 0.7, but for countries ranked 50th to 100th it was 0.57.
The average for charity is slightly higher than the average for happiness.
Next, let's add the GINI index, which is an inequality index for each country.
There is a table on the Wikipedia page, import it.
About Gini Coefficient:
In economics, the Gini coefficient, also known as the Gini index or Gini ratio, is a measure of statistical dispersion intended to represent the income inequality, the wealth inequality, or the consumption inequality within a nation or a social group. It was developed by Italian statistician and sociologist Corrado Gini.
List of countries by income equality:
This is a list of countries or dependencies by income inequality metrics, including Gini coefficients. The Gini coefficient is a number between 0 and 1, where 0 corresponds with perfect equality (where everyone has the same income) and 1 corresponds with perfect inequality (where one person has all the income—and everyone else has no income).
Income distribution can vary greatly from wealth distribution in a country (see List of countries by wealth inequality). Income from black market economic activity is not included and is the subject of current economic research.
Load from wikipedia
gini_url = 'https://en.wikipedia.org/wiki/List_of_countries_by_income_equality'
df = pd.read_html(gini_url)[1]
gi = df.droplevel(level=1, axis=1)
gi = gi.iloc[1:-1,[0, -2]]
gi.columns = ['country','rank']
gi = gi.sort_values('rank')
Add gini index ranking
gi['gini'] = range(1, len(gi) + 1)
gini = gi[['country','gini']]
replace_strings_in_dataframe(gini, replacements)
| country | gini | |
|---|---|---|
| 49 | Faroe Islands | 1 |
| 67 | Iceland | 2 |
| 135 | Slovakia | 3 |
| 136 | Slovenia | 4 |
| 38 | Czech Republic | 5 |
| ... | ... | ... |
| 26 | Central African Republic | 165 |
| 168 | Zambia | 166 |
| 144 | Suriname | 167 |
| 108 | Namibia | 168 |
| 138 | South Africa | 169 |
169 rows × 2 columns
df = df_give_happy_gdp.merge(gini, on='country', how='left')
df.gini = df.gini.fillna(df.gini.mean())
df_h = df['gini']
df['norm_gini'] = 1 - (df_h - df_h.min()) / (df_h.max() - df_h.min())
df_give_happy_gdp_gini = df.copy()
Sort by lowest Gini index rank, that is, by lowest income disparity.
df = df_give_happy_gdp_gini
sort_by = 'norm_gini'
xlabel = 'Gini Index Rank'
title = 'Happiness and Charitiable Total Score Rank Top 50 sort by Gini Index'
plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title)
And the average total score for happiness and charity came out to be 0.64. This is lower than the GDP ranking.
The difference between the average level of happiness and the average level of charity was 0.06 points.
Let's also add the Global Corruption Index.
url_cpi = 'https://en.wikipedia.org/wiki/Corruption_Perceptions_Index'
df = pd.read_html(url_cpi)[4]
cp = df.droplevel(level=0, axis=1)
cp = cp.iloc[0:,[1,0]]
cp.iloc[:,1] = cp.iloc[:,1].replace('—', np.nan).astype(float)
cp.columns = ['country','cpi']
cp = cp.sort_values('cpi')
replace_strings_in_dataframe(cp, replacements)
| country | cpi | |
|---|---|---|
| 0 | Denmark | 1.0 |
| 1 | New Zealand | 2.0 |
| 2 | Finland | 2.0 |
| 3 | Norway | 4.0 |
| 4 | Singapore | 5.0 |
| ... | ... | ... |
| 177 | Venezuela | 177.0 |
| 179 | South Sudan | 178.0 |
| 178 | Syria | 178.0 |
| 180 | Somalia | 180.0 |
| 38 | Brunei Darussalam | NaN |
181 rows × 2 columns
df = df_give_happy_gdp_gini.merge(cpi, on='country',how='left')
df.cpi = df.cpi.fillna(df.cpi.mean())
df_h = df['cpi']
df['norm_cpi'] = 1 - (df_h - df_h.min()) / (df_h.max() - df_h.min())
df_give_happy_gdp_gini_cpi = df.copy()
df = df_give_happy_gdp_gini_cpi
sort_by = 'norm_cpi'
xlabel = 'CPI Rank'
title = 'Happiness and Charitiable Total Score Rank Top 50 sort by Corruption Perceptions Index'
plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title)
The average value was 0.72.
The average level of happiness was 0.4, and the average level of charity was 0.32, with the difference increasing to 0.08.
Less corruption seems to make people happier and more charitable, but the increase in happiness appears to be large.
Large economies such as Hong Kong, Japan, and South Korea stand out for their low scores despite having low levels of corruption.
df[:11].norm_ttl.mean()
0.8773524156028142
Surprisingly, the top 11 countries all exceeded the average value of 0.73, and the average was 0.87.
So what does a country's average life expectancy have to do with happiness and philanthropy? Let's try importing the data.
Wikipedia has tables for the World Bank Group, United Nations, World Health Organization, and CIA World Factbook.
First, combine each into a data frame.
This time we will use data from World Health Organization.
url_life = 'https://en.wikipedia.org/wiki/List_of_countries_by_life_expectancy'
df = pd.read_html(url_life)
le_wb = df[0].droplevel(level=0,axis=1)
le_wb = le_wb.iloc[1:,:2]
le_wb.columns = ['country', 'wb']
le_wb.head()
| country | wb | |
|---|---|---|
| 1 | Hong Kong SAR, China | 85.49 |
| 2 | Macao SAR, China | 85.40 |
| 3 | Japan | 84.45 |
| 4 | Switzerland | 83.85 |
| 5 | South Korea | 83.53 |
le_un = df[1].droplevel(level=0,axis=1)
le_un = le_un.iloc[1:,:2]
le_un.columns = ['country', 'un']
le_un.head()
| country | un | |
|---|---|---|
| 1 | Hong Kong SAR, China | 84.9 |
| 2 | Japan | 84.6 |
| 3 | Australia | 83.4 |
| 4 | Switzerland | 83.8 |
| 5 | Malta | 82.5 |
le_who = df[2].droplevel(level=0, axis=1)
le_who = le_who.iloc[1:,:2]
le_who.columns = ['country','who']
le_who.head()
| country | who | |
|---|---|---|
| 1 | Japan | 84.3 |
| 2 | Switzerland | 83.4 |
| 3 | South Korea | 83.3 |
| 4 | Singapore | 83.2 |
| 5 | Spain | 83.2 |
In data of CIA Myanmar called Burma, and West Bank meaning is palestine area.
le_cia = df[3]
le_cia = le_cia.iloc[:227,:2]
le_cia.columns = ['country', 'cia']
le_cia.head()
| country | cia | |
|---|---|---|
| 0 | Monaco | 89.52 |
| 1 | Singapore | 86.35 |
| 2 | Macau | 84.98 |
| 3 | Japan | 84.83 |
| 4 | San Marino | 83.86 |
le_list = [le_wb, le_who, le_un, le_cia]
for i in le_list:
replace_strings_in_dataframe(i, replacements)
lex = merge_dataframes(le_list)
missing_country_data(lex)
['Northern Cyprus', 'Somaliland Region']
Before combining the data frames, fill the data for Northern cyprus and Somaliland as they are missing.
Data for Somaliland is from UNICEF, and data for Northern Cyprus is from the BBC.
fill_data = {'country': ['Northern Cyprus', 'Somaliland Region'],
'additional': [(79+83)/2, (55+57)/2],}
fill = pd.DataFrame(fill_data)
lex = pd.concat([lex, fill], ignore_index=True)
lex = lex.set_index('country')
lex['mean'] = lex.T.mean().T
lexlex = lex.reset_index().sort_values('mean',ascending=False)
lex['life_ex'] = range(1, lex.shape[0]+1)
lex= lex.iloc[:,[0,-1]]
lex
| country | life_ex | |
|---|---|---|
| 207 | Monaco | 1 |
| 1 | Macao | 2 |
| 0 | Hong Kong | 3 |
| 2 | Japan | 4 |
| 6 | Singapore | 5 |
| ... | ... | ... |
| 201 | Chad | 236 |
| 196 | Somalia | 237 |
| 239 | Somaliland Region | 238 |
| 199 | Lesotho | 239 |
| 198 | Central African Republic | 240 |
240 rows × 2 columns
df = df_give_happy_gdp_gini_cpi.merge(lex, on='country', how= 'left')
df_h = df['life_ex']
df['norm_life_ex'] = 1 - (df_h - df_h.min()) / (df_h.max() - df_h.min())
df_give_happy_gdp_gini_cpi_life = df.copy()
df = df_give_happy_gdp_gini_cpi_life
sort_by = 'norm_life_ex'
xlabel = 'Life Expectancy Rank'
title = 'Happiness and Charitiable Total Score Rank Top 50 sort by Life Expectancy'
plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title)
Monaco ranks first in terms of average life expectancy, followed by Macau, but these two items are not present in the giving index or happieness rank data.
The average value is 0.73, which is a high value.
The average happiness rating was 0.41 and the average charity rating was 0.32, a difference of 0.09.
People seem to be significantly happier as they live longer on average.
However, low scores stand out in countries with high life expectancy, such as Hong Kong, Japan, and South Korea in top 10 countries.
Speaking of charity work, one can think of fundraising in times of disaster.
Will donations be made in countries that are prone to natural disasters or surrounding countries, making them more charitable?
This United Nations Office for Disaster Risk Reduction website provides detailed information on natural disaster risks.
Add natural disaster risk data from wikipedia.
url_ndr = 'https://en.wikipedia.org/wiki/List_of_countries_by_natural_disaster_risk'
df = pd.read_html(url_ndr)[0]
ndr = df.iloc[:,[1,0]]
ndr.columns = ['country','ndr']
replace_strings_in_dataframe(ndr, replacements)
| country | ndr | |
|---|---|---|
| 0 | Andorra | 1 |
| 1 | Monaco | 1 |
| 2 | San Marino | 2 |
| 3 | São Tomé and Príncipe | 3 |
| 4 | Luxembourg | 4 |
| ... | ... | ... |
| 188 | Mexico | 181 |
| 189 | Colombia | 182 |
| 190 | Indonesia | 183 |
| 191 | India | 184 |
| 192 | Philippines | 185 |
193 rows × 2 columns
df = df_give_happy_gdp_gini_cpi_life.merge(ndr, on='country', how='left')
df.ndr = df.ndr.fillna(df.ndr.max())
df_h = df['ndr']
df['norm_ndr'] = 1 - (df_h - df_h.min()) / (df_h.max() - df_h.min())
df_give_happy_gdp_gini_cpi_life_ndr = df.copy()
df = df_give_happy_gdp_gini_cpi_life_ndr
sort_by = 'norm_ndr'
xlabel = 'Natural Disaster Risk Rank'
title = 'Happiness and Charitiable Total Score Rank Top 50 sort by Natural Disaster (Low) Risk'
plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title)
The mean value was 0.59, and the difference between happiness and benevolence narrowed to just 0.01.
The low risk of natural disasters may have the effect of making benevolence and happiness similar.
df = df_give_happy_gdp_gini_cpi_life_ndr
sort_by = 'norm_ndr'
xlabel = 'Natural Disaster Risk Rank'
title = 'Natural Disaster Risk Ranking 51 to 100'
plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title, 51, 100)
df = df_give_happy_gdp_gini_cpi_life_ndr
sort_by = 'norm_ndr'
xlabel = 'Natural Disaster Risk Rank'
title = 'Natural Disaster Risk Ranking 101 to Last'
plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title, 101, -1)
As the risk of natural disasters increased, the gap between benevolence and happiness further decreased.
For those ranked 51st and below, charity was 0.01 points higher than happiness.
For those ranked 101st and below, the level of charity increased to 0.03 points higher than happiness.
Countries that are prone to natural disasters seem to be more benevolent.
So what about the impact of population? Import and compare data.
url_pop = 'https://en.wikipedia.org/wiki/List_of_countries_and_dependencies_by_population'
df = pd.read_html(url_pop)
After the integration, Somaliland data will be missing, so we will add it as new data. The numbers are taken from the UNICEF pdf file.
Since China and India account for 34% of the total population, we normalize them by rank rather than population.
po = df[0].iloc[1:,1:3]
po.columns = ['country','num']
new_data = {'country': ['Somaliland Region'], 'num': [4294135]}
new = pd.DataFrame(new_data)
po = pd.concat([po,new], ignore_index=True)
replace_name = {r'\s\(\w+\)': ''}
replace_strings_in_dataframe(po, replace_name)
replace_strings_in_dataframe(po, replacements)
po = po.sort_values('num',ascending=False)
po.insert(1, 'pop', range(1, len(po) + 1))
pop = po.iloc[:,[0,1]]
df = df_give_happy_gdp_gini_cpi_life_ndr.merge(pop, on='country', how='left')
df_h = df['pop']
df['norm_pop'] = 1 - (df_h - df_h.min()) / (df_h.max() - df_h.min())
df_give_happy_gdp_gini_cpi_life_ndr_pop = df.copy()
df = df_give_happy_gdp_gini_cpi_life_ndr_pop
sort_by = 'norm_pop'
xlabel = 'Population Rank'
title = 'Happiness and Charitiable Total Score Rank Top 50 sort by Population'
plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title)
The average value is now 0.55. This is the lowest average ever. Interestingly, in the top 50 most populous countries, charity was 0.05 points higher than happiness.
A larger population suggests that people are more charitable.
In recent years, rising suicide rates have become a major issue in developed countries.
It is known that the incidence of winter depression is higher in northern countries with higher latitudes and shorter daylight hours. Depression is one of the causes of suicide.
How does a country's suicide rate affect happiness and charity?
url_suc = 'https://en.wikipedia.org/wiki/List_of_countries_by_suicide_rate'
df = pd.read_html(url_suc)
suc_c = df[1].iloc[:183,:2]
suc_c.columns = ['country', 'new']
suc_c.loc[:,'country'] = suc_c['country'].replace(r'\s\*','',regex=True)
suc_c = replace_strings_in_dataframe(suc_c, replacements)
suc_c
| country | new | |
|---|---|---|
| 0 | Afghanistan | 6.0 |
| 1 | Albania | 3.7 |
| 2 | Algeria | 2.6 |
| 3 | Angola | 12.6 |
| 4 | Antigua and Barbuda | 0.3 |
| ... | ... | ... |
| 178 | Venezuela | 2.1 |
| 179 | Vietnam | 7.2 |
| 180 | Yemen | 7.1 |
| 181 | Zambia | 14.4 |
| 182 | Zimbabwe | 23.6 |
183 rows × 2 columns
suc_r = df[6].iloc[:,[0,3]]
suc_r.columns = ['country', 'old']
suc_r.loc[:,'country'] = suc_r['country'].replace(r'\s\*|\([\w\s]+\)','',regex=True)
suc_r = replace_strings_in_dataframe(suc_r, replacements)
There are two types of data for Taiwan, the government-published value and the estimated value, so we will use the average value.
duplicates = suc_r[suc_r.duplicated(subset='country', keep=False)]
duplicates
| country | old | |
|---|---|---|
| 9 | Taiwan | 20.1 |
| 17 | Taiwan | 16.4 |
merged_suc_r = suc_r.groupby('country')['old'].mean().reset_index()
merged_suc_r[95:100]
| country | old | |
|---|---|---|
| 95 | Switzerland | 11.20 |
| 96 | Syria | 0.10 |
| 97 | São Tomé and Príncipe | 0.90 |
| 98 | Taiwan | 18.25 |
| 99 | Tajikistan | 2.60 |
Since there are countries such as Mexico that have values in both dataframes. Add a new column 'Rate' and make sure its content is the value of the 'new' column if it has data, or the value of the 'old' column if the 'new' column is NaN.
suc = suc_c.merge(merged_suc_r, on='country', how='outer')
suc['rate'] = suc['new'].fillna(suc['old'])
suc = suc.iloc[:,[0,-1]]
suc.sort_values('rate')
| country | rate | |
|---|---|---|
| 186 | Saint Kitts and Nevis | 0.00 |
| 13 | Barbados | 0.30 |
| 4 | Antigua and Barbuda | 0.30 |
| 66 | Grenada | 0.60 |
| 138 | Saint Vincent and the Grenadines | 1.00 |
| ... | ... | ... |
| 87 | Kiribati | 30.60 |
| 55 | Eswatini | 40.50 |
| 70 | Guyana | 40.90 |
| 184 | Greenland | 53.34 |
| 93 | Lesotho | 87.50 |
188 rows × 2 columns
The data shows that Lesotho, which ranks last, stands out.
If we normalize the data as it is, the data will be biased, so we will rank it first.
suc = suc.sort_values('rate')
suc.insert(1, 'suc', range(1, len(suc)+1))
suc = suc.iloc[:,:2]
suc
| country | suc | |
|---|---|---|
| 186 | Saint Kitts and Nevis | 1 |
| 13 | Barbados | 2 |
| 4 | Antigua and Barbuda | 3 |
| 66 | Grenada | 4 |
| 138 | Saint Vincent and the Grenadines | 5 |
| ... | ... | ... |
| 87 | Kiribati | 184 |
| 55 | Eswatini | 185 |
| 70 | Guyana | 186 |
| 184 | Greenland | 187 |
| 93 | Lesotho | 188 |
188 rows × 2 columns
df = df_give_happy_gdp_gini_cpi_life_ndr_pop.merge(suc, on='country', how='left')
df.suc = df.suc.fillna(df.suc.mean())
df_h = df['suc']
df['norm_suc'] = 1 - (df_h - df_h.min()) / (df_h.max() - df_h.min())
df_give_happy_gdp_gini_cpi_life_ndr_pop_suc = df.copy()
df = df_give_happy_gdp_gini_cpi_life_ndr_pop_suc
sort_by = 'norm_suc'
xlabel = 'Suicide Rate'
title = 'Happiness and Charitiable Total Score Rank Top 50 sort by Suicide Rate'
plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title)
When the suicide rate is low, the total score is low at 0.54, and benevolence exceeds happiness by 0.02 points.
It doesn't seem to have much to do with happiness.
df = df_give_happy_gdp_gini_cpi_life_ndr_pop_suc
sort_by = 'norm_suc'
xlabel = 'Suicide Rate'
title = 'Happiness and Charitiable Total Score Rank Top 50 sort by Suicide Rate'
plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title, 51,110)
Looking at 50 and below, the total score increased to 0.7 points. This is an unexpected result.
Add the free index as the last data frame.
Note that this is different from the democracy index.
url_free = 'https://en.wikipedia.org/wiki/Freedom_in_the_World'
df = pd.read_html(url_free)
w = df[0]
t = df[1]
free_w = w.iloc[1:,[0,5,9,13,17]].droplevel(level=1, axis=1)
free_t = t.iloc[1:,[0,19,23]].droplevel(level=1, axis=1)
free_t.columns = free_t.columns.str.replace(r'\[\d+\]','',regex=True)
fdm = pd.concat([free_w, free_t])# w
fdm = fdm.rename(str.lower, axis='columns')
replace_dic = {
r'\(.+\).?':'',
r'.?\*.?':'',
}
replace_strings_in_dataframe(fdm, replace_dic)
replace_strings_in_dataframe(fdm, replacements)
fdm.country = fdm.country.str.strip()
fdm = fdm.set_index('country')
fdm['freedom'] = fdm.T.mean().T
fdm = fdm.sort_values('freedom',ascending=False).iloc[:,-1].reset_index()
Puerto Rico has no freedom world points. However, since this is the United States, enter the same score as your united stats.
pr_row = fdm[fdm["country"] == "Puerto Rico"]
us_row = fdm[fdm["country"] == "United States"]
fdm.loc[pr_row.index, ["freedom"]] = fdm.loc[us_row.index, ["freedom"]].values
fdm[fdm["country"] == "Puerto Rico"]
| country | freedom | |
|---|---|---|
| 210 | Puerto Rico | 85.0 |
Since there are multiple countries with the same score, we will directly normalize the scores without ranking them.
df = df_give_happy_gdp_gini_cpi_life_ndr_pop_suc.merge(fdm, on='country', how='left')
df_h = df['freedom']
df['norm_freedom'] = (df_h - df_h.min()) / (df_h.max() - df_h.min())
df_give_happy_gdp_gini_cpi_life_ndr_pop_suc_freedom = df.copy()
df = df_give_happy_gdp_gini_cpi_life_ndr_pop_suc_freedom
sort_by = 'norm_freedom'
xlabel = 'Freedom in the World Rank'
title = 'Happiness and Charitiable Total Score Rank Top 50 sort by Freedom Rank'
plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title)
The average score for the top 50 countries was 0.74, the highest ever.
Happiness is 0.42, charity is 0.32, and the gap between them is 0.1 point, which is also the highest difference.
It seems that having a high degree of freedom has a significant effect on happiness. However, there is no effect on charity.
Let's take a look at the correlation between the features of the data frames added so far.
data = df_give_happy_gdp_gini_cpi_life_ndr_pop_suc_freedom.set_index('country')
cols = [col for col in data.columns if 'norm' in col]
df = data.loc[:,cols[:3]+cols[6:]]
fig = plt.subplots(figsize=(9, 8))
sns.heatmap(df.corr(), annot=True,cmap='PuBu_r')
plt.show()
White colors have a stronger correlation, and blue colors have an inverse correlation.
The strongest correlation with happiness is life expectancy, with a correlation rate of 0.79, which is quite strong. It seems that as life expectancy increases, people become happier. The next most correlated factor is the corruption index, which is 0.7. It seems that when corruption decreases, people become happier. Freedom rank also correlates with happiness.
The correlation with natural disaster risk is 0.11, so it seems that even if the risk is low, it does not have much of a relationship with happiness. What is interesting is the population. There is an inverse correlation with happiness at -0.19, indicating that happiness decreases when the population is large.
Happiness is the most correlated factor with Giving. However, the correlation is weak at 0.33. There is also a weak relationship with GDP at 0.3. As people become more financially affluent, philanthropy appears to increase. I speculated that people would become more charitable if there were many natural disasters. In the top 50 countries, it seems that the level of charity has increased, partly because the gap with happiness has narrowed, but at 0.014, there is almost no relationship. And the population bonus seems to have little to do with it. Freedom and life expectancy have very weak correlations with benevolence.
What I would like to draw attention to is the inverse correlation between natural disaster risk and population, which is the most significant -0.49. This may be the result of a tendency to avoid disaster areas as a place of residence. There is also an inverse relationship with GDP, with fewer natural disasters leading to more economic development.
Population appears to have an inverse relationship as a factor. Suicide rates are uncorrelated with almost any factor.
Surprisingly, the results showed a strong relationship between average life expectancy and corruption index correlation are 0.74.
Note about is how to fill in missing values.
GDP, population, and life expectancy are easy to fill in because there is abundant data, but missing values for other items such as the Gini index, corruption index, and natural disaster risk are substituted with average values because of limited sources. .
Please also consider that they are not treated as unranked and therefore have some bonuses.
Let's take a look at the overall rankings of each country. On a scale of 0 to 1, points are 1 for the 1st rank and 0 for the lowest rank.
df= df_give_happy_gdp_gini_cpi_life_ndr_pop_suc_freedom.set_index('country')
cols = [col for col in df.columns[1:] if 'norm' in col]
df_columns = cols[:3]+cols[6:]
df = df.loc[:,df_columns]
df['total'] = df.sum(axis=1)
df = df.sort_values('total')
columns_to_plot = df.columns[:-1]
plt.figure(figsize=(10, 50))
x = df.index
y = df.total
plt.barh(x, y, label = ' ', color= 'gray', alpha=.5)
custom_palette = ["#f5e66b", # giving
"#f482bd", # happiness
"#8de5ec", # dgp
"#e696f0", # gini
"#b39640", # cpi
"#f68189", # life
"#a7c1a9", # ndr
"#96acf0", # pop
"#b3b3b3", # suc
"#e2fab9", # free
]
sns.set_palette(custom_palette)
for i, column in enumerate(columns_to_plot):
color = sns.color_palette()[i]
left = df[columns_to_plot[:i]].sum(axis=1)
plt.barh(x, df[column], label=' ', color=color, left=left)
for j, val in enumerate(df[column]):
total = y.iloc[j]
plt.text(total+0.05, j, f'{total:.2f}', va='center',ha='left', color='darkgray')
for g in g20:
if g in x:
plt.barh(g, df[df.index==g].total, color='gray', alpha=0.4)
plt.plot([], [], ' ', label=" ")
plt.ylim(-1, len(df)-.1)
plt.xlabel('Total Normalized Value',fontsize=14)
plt.ylabel('Country', fontsize=14)
plt.title('Stacked Total Scores Rank of Happiness Index {} countries'.format(len(df)),fontsize=18)
label_list = ["Number is Total Score",
'with Dark Color: G20 Member',
'Happiness Index',
'Giving Index',
'Gross Domestic Product',
'Gini Index',
'Corruption Perceptions Index',
'Liefe Expectancy',
'Natural Disaster Risk',
'Population',
'Suicide Rate',
'Freedom In The World']
L = plt.legend()
for i, l in enumerate(label_list):
L.get_texts()[i].set_text(label_list[i])
plt.show()
After normalizing each rank, the highest rank of the total score is the Denmark.
The top spots are mostly filled by Europe.
Brazil ranks highest in South America, while Singapore ranks highest in Asia.
The second half of the list is dominated by African countries.
The data contained many missing values, and many countries were not ranked.
Normalization also includes filling in missing values with no data using average values from past ranks.
%%html
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.4.2/css/all.min.css">