Exploring relationship between Happiness , Givingness and Country-factors¶

In this project find datasets on the web and visualize these datasets.

Although it is extremely difficult to quantify human psychology and feelings, we will use data on happiness and charity as reference indicators. Add factor data to your dataset and explore relationships between each piece of data.

The dataset is based on country names. As there may be discrepancies in names between reference source data, we have unified them to the extent possible. Please note that not all countries in the world are included.

  • Regions and regional categories covered by the dataset

    • Country where data exists, unit is country
  • Questions about domain categories and regions of interest

    • How do happiness and charity levels vary by country factors?

Data Sources¶

  • World Giving Index Wikipedia

    • Charities Aid Foundation Web Site
  • World Happiness Report Wikipedia
    • World Happiness Report Web Site
  • G20 Wikipedia
  • GDP Wikipedia)
    • Unicef Somalirand PDF file
  • Gini Index Wikipedia
  • Corruption Perceptions Index data Wikipedia
    • Transparency International Web Site
  • Life Expectancy Wikipedia

    • WHO World Health Statistics
    • CIA The World Factbook

    • World Bank : Natural Disaster Hotspots A Global Risk Analysis PDF file

    • Unicef Somalirand PDF file

    • BBC Northern Cyprus Web Site

  • IMF World Economic Outlook
  • Natural Disaster Risk Wikipedia
    • Bündnis Entwicklung Hilft : The WorldRiskReport PDF file
  • Population Wikipedia
  • Suicide Rate Wikipedia
    • worldpopulationreview web site
  • Freedom in thw World Wikipedia

import libraries¶

import numpy as np
import pandas as pd
import seaborn as sns
import scipy.stats as stats
import plotly.express as px
import matplotlib.pyplot as plt

from matplotlib.cm import get_cmap
from matplotlib.patheffects import withStroke

import warnings
warnings.simplefilter('ignore', FutureWarning)

import plotly.express and arrange DataFrame to plot ranking

Define urls¶

url_G20 = 'https://en.wikipedia.org/wiki/G20'

url_WGI = 'https://en.wikipedia.org/wiki/World_Giving_Index'

url_WHR = 'https://en.wikipedia.org/wiki/World_Happiness_Report'

G20¶

Extract country names from G20 Member without African Unioin and European Union

g20 = pd.read_html(url_G20)[3]
g20 = g20['Member'][:19]

g20
0          Argentina
1          Australia
2             Brazil
3             Canada
4              China
5             France
6            Germany
7              India
8          Indonesia
9              Italy
10             Japan
11            Mexico
12       South Korea
13            Russia
14      Saudi Arabia
15      South Africa
16            Turkey
17    United Kingdom
18     United States
Name: Member, dtype: object

Giving Index Data Frame¶

Create giving data frame

Nan values are replaced by the average value for each country for later normalization.

df = pd.read_html(url_WGI)[1]
df.columns = df.columns.str.replace(r'(?<=\d{4}).*', '', regex=True)

Fill Nan values

giv = df.set_index('Country')
name_mapping = {'Macedonia':'North Macedonia'}
giv = giv.rename(index=name_mapping)

giving = giv.T.fillna(giv.T.mean(axis=0), axis=0).T
giving.sample(5)
2022 2021 2019 2018 2017 2016 2015 2014 2013
Country
Spain 99.0 100.0 58.0 54.0 71.0 79.0 58.0 62.0 57.0
Zambia 9.0 17.0 32.0 21.0 18.0 63.0 51.0 47.0 45.0
France 100.0 106.0 66.0 72.0 67.0 81.0 74.0 90.0 77.0
Argentina 21.0 74.0 75.0 94.0 83.0 83.0 108.0 77.0 78.0
Serbia 27.0 48.0 123.0 129.0 132.0 135.0 118.0 124.0 123.0

Merge World_Giving_Index dataframe and G20 country name

Replace rank[number] strings

Normalize rank data

df = giving[giving.columns[::-1]]
df = df.loc[g20]
give_20 = 1 - (df - df.min())/(df.max() - df.min())
give_20.head()
2013 2014 2015 2016 2017 2018 2019 2021 2022
Country
Argentina 0.416667 0.401575 0.251748 0.413043 0.404412 0.340426 0.408 0.353982 0.829060
Australia 0.954545 0.960630 0.972028 0.992754 0.970588 0.992908 0.976 0.964602 0.974359
Brazil 0.583333 0.566929 0.580420 0.608696 0.602941 0.489362 0.416 0.530973 0.854701
Canada 0.992424 0.984252 0.979021 0.971014 0.963235 0.907801 0.960 0.699115 0.940171
China 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000 0.168142 0.589744

Transpose dataframe¶

sort dataframe rank and year, to make default 2013 rank
in these case need to add kind option

t_ = give_20.stack().reset_index()
t_.columns = ['country', 'year', 'rank']

t_rank = t_.sort_values('rank', ascending=False).sort_values('year',ascending=True,kind='stable')
t_rank.head()
country year rank
162 United States 2013 1.000000
27 Canada 2013 0.992424
153 United Kingdom 2013 0.962121
9 Australia 2013 0.954545
72 Indonesia 2013 0.878788

Plot data¶

fig = px.bar(t_rank,
                x = "rank",
                y = "country",
                color = 'country',
                animation_frame = "year",
                title="G20 Countries Charitable Trend from 2013 to 2022"
                )

fig.update_layout(margin=dict(l=0, r=0, t=30, b=0))
fig.layout.updatemenus[0].buttons[0].args[1]["frame"]["duration"] = 800

fig.show()

In the case below, it use receplotly to move the y-axis, so the rank order will always preserved.

from raceplotly.plots import barplot

my_raceplot = barplot(t_,
                      item_column='country',
                      value_column='rank',
                      time_column='year',
                      top_entries = 20,
                     )

my_raceplot.plot(title = 'Changes in the Giving index Ranks of G20 countries from 2013 to 2022',
                 item_label = 'Top 20 Giveng Countries',
                 value_label = 'Rank',
                 frame_duration = 1000)

Show rank in line chart

fig = px.line(t_,
              x = "year",
              y = "rank",
              color = 'country',
              markers = True
                )

fig.show()

World Happiness Rank Data Frame¶

Load Data from World Happiness Report extract 2023 report

There is data for only 20 countries in 2022, and no data in 2021.
The 2023 report includes average happiness scores from 2020 to 2022.
Hence it will use data from 2023.

Load data¶

df = pd.read_html(url_WHR)[12:-1]

Make dataFrame from each table

c_2023 = df[0]
# c_2022 = df[2]
c_2020 = df[4]
c_2019 = df[6]
c_2018 = df[8]
c_2017 = df[10]
c_2016 = df[13]
c_2013 = df[15]

Wikipedia has a useful collection of information, but there are some years where data is missing.

Data for 2022, 2021, and 2015 are obtained from the World Happiness Report web page.

There is also 2012, add it.

y_2022 = 'Appendix_2_Data_for_Figure_2.1.xls'
y_2021 = 'DataForFigure2.1WHR2021C2.xls'
y_2015 = 'Chapter2OnlineData_Expanded-with-Trust-and-Governance.xlsx'
y_2012 = '2012.xlsx'
c_2022 = pd.read_excel(y_2022)
c_2022 = c_2022.iloc[:,[0,1]]
c_2021 = pd.read_excel(y_2021)
c_2021['2021'] = range(1, c_2021.shape[0] +1)
c_2021 = c_2021.iloc[:,[-1,0]]
ex = pd.ExcelFile(y_2015)
c_2015 = ex.parse(ex.sheet_names[1])
c_2015.columns = c_2015.iloc[2]
c_2015 = c_2015.iloc[3:,[1]].reset_index(drop=True)
exl = pd.ExcelFile('2012.xlsx')
c_2012 = exl.parse(exl.sheet_names[1])
c_2012 = c_2012.iloc[:,[0]]

Edit dataframe

c_2023.columns = ['2023','country']
c_2023 = c_2023.iloc[:,[1,0]]

c_2022.columns = ['2022','country']
c_2021.columns = ['2021','country']

c_2020 = c_2020.iloc[:,:2]
c_2020.columns = ['2020','country']

c_2019 = c_2019.iloc[:,:2]
c_2019.columns = ['2019','country']

c_2018 = c_2018.iloc[:,:2]
c_2018.columns = ['2018','country']

c_2017 = c_2017.iloc[:,[0,2]]
c_2017.columns = ['2017','country']

c_2016 = c_2016.iloc[:,:2]
c_2016.columns = ['2016','country']

c_2015['2015'] = range(1, c_2015.shape[0] +1)
c_2015 = c_2015.iloc[:,[1,0]]

c_2013 = c_2013.iloc[:,:2]
c_2013.columns = ['2013','country']

c_2012['2012'] = range(1, c_2012.shape[0] +1)
c_2012.columns = ['country', '2012']
c_2012 = c_2012.iloc[:,[1,0]]

Replace Country Name Function¶

Replace and unify the names so that they have the same country name

def replace_strings_in_dataframe(df, replacements, col='country'):
    
    df.loc[:,col] = df[col].replace(replacements, regex=True)
    return df

Replace pattern dictionary¶

replacements = {
    '&': 'and',
    r'(?<=\w)\*': '',
    'region': 'Region',
    r'(?<=\w)\[\w+?\]': '',
    'Czechia':'Czech Republic',
    'Slovak Republic': 'Slovakia',
    'Trkiye': 'Turkey',
    'Burma':'Myanmar',
    'Viet.+$':'Vietnam',
    'The Gambia': 'Gambia',
    'Kyrgyz Republic': 'Kyrgyzstan',
    r'^Korea': 'South Korea',
    'Lao P.D.R.': 'Laos',
    "Cote.+$|Côte.+$": "Ivory Coast",
    'North Cyprus':'Northern Cyprus',
    r'^Macedonia$': 'North Macedonia',
    'Eswatini.+$|Swaziland': 'Eswatini',
    'Taiwan.+$': 'Taiwan',
    'Maca.+$|Maca':'Macao',
    r'^Somaliland$': 'Somaliland Region',
    "People's Republic of China": 'China',
    'Hong Kong.+$': 'Hong Kong',
    'United States.+$|USA': 'United States',
    r'Palestin.+$|West Bank':'Palestinian Territory',
    r'Congo, Rep.+$|^Republic of (the )?Congo|Congo \(Brazzaville\)': 'Congo',
    r'Congo, Democratic Republic of|Democratic Congo|Congo \(Kinshasa\)|DR Congo': 'Democratic Republic of the Congo',
}
df_name_list = [c_2023,c_2022,c_2021,c_2020,c_2019,c_2018,c_2017,c_2016,c_2015,c_2013,c_2012]
for i in df_name_list:
    replace_strings_in_dataframe(i, replacements)

Merge each years data frame

Define merge function

The Nan value will be out of rank and will be replaced by the max value here for normalization later.

Merge all year dataframe¶

def merge_dataframes(dfs, on_key='country'):

    merged_df = dfs[0]
    for df in dfs[1:]:
        merged_df = merged_df.merge(df, on=on_key, how='outer')

    return merged_df
df = merge_dataframes(df_name_list)
df = df[df.columns[::-1]].set_index('country')
df = df.apply(pd.to_numeric, errors='coerce')
# happiness = happiness.fillna(happiness.max())
happiness = df.T.fillna(df.T.mean(axis=0), axis=0).T
indexes_to_drop = ['World', 'Europe','xx']

happiness = happiness.drop(indexes_to_drop)
happiness.tail(5)
2012 2013 2015 2016 2017 2018 2019 2020 2021 2022 2023
country
Somaliland Region 111.0 99.666667 91.0 97.0 99.666667 99.666667 99.666667 99.666667 99.666667 99.666667 99.666667
Oman 22.5 23.000000 22.0 22.5 22.500000 22.500000 22.500000 22.500000 22.500000 22.500000 22.500000
Djibouti 95.0 110.500000 126.0 110.5 110.500000 110.500000 110.500000 110.500000 110.500000 110.500000 110.500000
Guyana 45.0 45.000000 45.0 45.0 45.000000 45.000000 45.000000 45.000000 45.000000 45.000000 45.000000
Cuba 69.0 69.000000 69.0 69.0 69.000000 69.000000 69.000000 69.000000 69.000000 69.000000 69.000000

Select G20 countries and normalize their ranks.

df = happiness.loc[g20]
happy_20 = 1 - (df - df.min())/(df.max() - df.min())
happy_20.head()
2012 2013 2015 2016 2017 2018 2019 2020 2021 2022 2023
country
Argentina 0.682243 0.809760 0.776786 0.821429 0.852174 0.825397 0.709924 0.669173 0.640625 0.637097 0.649123
Australia 0.962617 0.966915 0.955357 0.973214 0.982609 0.976190 0.984733 0.992481 1.000000 1.000000 1.000000
Brazil 0.813084 0.851117 0.901786 0.901786 0.869565 0.833333 0.824427 0.842105 0.812500 0.790323 0.675439
Canada 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 0.976562 0.975806 0.991228
China 0.000000 0.345740 0.294643 0.312500 0.373913 0.373016 0.358779 0.375940 0.429688 0.516129 0.543860
h_ = happy_20.stack().reset_index()
h_.columns = ['country', 'year', 'rank']

h_rank = h_.sort_values('rank', ascending=False).sort_values('year',ascending=True,kind='stable')
h_rank.head()
country year rank
33 Canada 2012 1.000000
11 Australia 2012 0.962617
198 United States 2012 0.943925
187 United Kingdom 2012 0.878505
55 France 2012 0.831776
fig = px.bar(h_rank,
                x = "rank",
                y = "country",
                color = 'country',
                animation_frame = "year",
                title="G20 Countries Happiness Trend from 2012 to 2023"
                )

fig.update_layout(margin=dict(l=0, r=0, t=50, b=0))
fig.layout.updatemenus[0].buttons[0].args[1]["frame"]["duration"] = 800

fig.show()

Merge Giving Index and Happiness Index dataframe¶

giving_2022 = giving.iloc[:,:1].reset_index()
giving_2022.columns = ['country','giving']
giving_2022
country giving
0 Indonesia 1.000000
1 Kenya 2.000000
2 United States 3.000000
3 Australia 4.000000
4 New Zealand 5.000000
... ... ...
154 Suriname 66.000000
155 Syria 43.666667
156 Trinidad and Tobago 16.500000
157 Turkmenistan 38.000000
158 Yemen 136.000000

159 rows × 2 columns

Extract 2023 form happiness rank dataframe

Merge Happiness Data Frame with Giving Data Frame¶

Merge two dataframes.
Specify outer to avoid deleting Nan when merge two dataframes.

normalize each rank value

calculate socores

happy = happiness[['2023']].reset_index()
happy.columns = ['country','happiness']
df =  happy.merge(giving_2022, on='country', how='outer')

Normalize each columns¶

Average values cannot be obtained because no past data exists. Therefore, replace it with the mean value.

df.giving = df.giving.fillna(df.giving.mean())
df_h = df['happiness']
df['norm_happiness'] = 1 - (df_h - df_h.min()) / (df_h.max() - df_h.min())

df_g = df['giving']
df['norm_giving'] = 1 - (df_g - df_g.min()) / ( df_g.max() - df_g.min())


df['diff'] = 1- abs(df['norm_happiness'] - df['norm_giving'])
df['ttl'] = df['norm_happiness'] + df['norm_giving']

df_give_happy = df.set_index('country')
df_give_happy = df_give_happy.apply(pd.to_numeric, errors='coerce')

df_give_happy['happiness'] = df_give_happy['happiness'].astype('int')
df_give_happy['giving'] = df_give_happy['giving'].astype('int')

Plot sort by Giving Index¶

sns.set()
sort_by = 'norm_giving'
df= df_give_happy.sort_values(sort_by, ascending=False)

df = df.iloc[:50]
hap = 'Happiness ranking'
giv = 'Giving ranking'
bet = 'Gap between happiness and charity'

x = df.index
y_hap = df['norm_happiness']
y_giv = df['norm_giving']

fig, ax = plt.subplots(figsize=(18, 7))
ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)
ax.spines['left'].set_visible(False)
ax.set_yticks([])

shadow = withStroke(linewidth=4, foreground='w')

plt.bar(x, y_giv, color='gray', alpha=.5, label='with Dark Color: G20 Member')

plt.bar(x, y_giv, color='#f5e66b', alpha=.9, width = 0.8, label=giv)

for g in g20:
    if g in x:
        plt.bar(g, df[df.index==g].norm_giving, color='gray', alpha=0.4)

plt.vlines(x, y_giv, y_hap, color='#f482bd', alpha = 1, label=bet)
plt.scatter(x, y_hap, color='#f482bd', alpha = 1, s = 300, label=hap, zorder=3).set_path_effects([shadow])

for i in range(len(x)):
    plt.annotate("{}".format(df['happiness'][i]),xy=(x[i], y_hap[i]), 
                 ha="center", va="center", color='w', fontsize=12)

plt.xlabel("Charitable Countries", fontsize = 14)
plt.ylabel("Charitable Ranking", fontsize = 14)
plt.title("Most Charitable Countries Rank with Happiness Rank Top 50", fontsize = 20)
plt.legend(shadow = True, fontsize = 12, loc = 'lower right', bbox_to_anchor=(1, .0))
plt.xticks(fontsize = 12)
fig.autofmt_xdate(rotation=45)

plt.show()

The graph shows that the countries with the highest philanthropy rankings do not necessarily have the highest levels of happiness.
The gray bar shows that countries other than the G20 are also passionate about charity.
It is also possible that charity demand may temporarily increase in the event of a natural disaster.

Change the sorting method and plot a graph in order of happiness level

Plot sort by Happiess index¶

sort_by = 'norm_happiness'
df = df_give_happy.sort_values(sort_by, ascending=False)
df = df.iloc[:50]

df.norm_giving = df.norm_giving.fillna(0)

hap = 'Happiness ranking'
giv = 'Giving ranking'
bet = 'Gap between happiness and charity'

x = df.index
y_hap = df['norm_happiness']
y_giv = df['norm_giving']

fig, ax = plt.subplots(figsize=(18, 7))

plt.bar(x, y_giv, color='gray', alpha=.5, label='with Dark Color: G20 Member')

plt.bar(x, y_giv, color='#f5e66b', alpha=.9, width = 0.8, label=giv)

for g in g20:
    if g in x:
        plt.bar(g, df[df.index==g].norm_giving, color='gray', alpha=0.4)

plt.vlines(x, y_giv, y_hap, color='#f482bd', label=bet)
plt.scatter(x, y_hap, color='#f482bd', s = 300, label=hap, zorder=3).set_path_effects([shadow])

for i in range(len(x)):
    plt.annotate("{}".format(df['happiness'][i]),xy=(x[i], y_hap[i]),
                 ha="center", va="center", color='w', fontsize=14)

plt.xlabel("Happiness Countries", fontsize=14)
plt.ylabel("World Happiness Rank", fontsize=14)
plt.title("Most Happiness Rank with Charitiable Rank Top 50", fontsize=18)
plt.legend(shadow=True, fontsize=12, loc='upper right', bbox_to_anchor=(1,1))
plt.xticks(fontsize=12)
fig.autofmt_xdate(rotation=45)

plt.show()

It's the same as the previous graph, just looking at it differently.
The smaller the difference, that is, the shorter the pink line, the stronger the correlation between happiness and charity. Let's take a look.
To plot this, sort by the diff column added during normalization.

sort_by = 'diff'
df= df_give_happy.sort_values(sort_by, ascending=False)
df = df.iloc[:50]

diff = 'Happiness Rank and Charitiable Rank Correlative Country'
diff_color = 'Countries (with Color : G20 Member)'

x = df.index
y_diff = df['diff']

fig, ax = plt.subplots(figsize=(18, 5))

plt.bar(x, y_diff, color='silver', alpha=0.5, width = 0.8, label=diff_color)

ax_1 = df.plot(kind='bar', label = diff)
plt.close(ax_1.figure)
for ticks in ax_1.xaxis.get_major_ticks():
    if ticks.label1.get_text() in list(g20):
        ax.patches[df.index.get_indexer([ticks.label1.get_text()])[0]].set_facecolor('deepskyblue')

plt.xlabel("Correlation Countries", fontsize=14)
plt.ylabel("Correlation Rank", fontsize=14)
plt.title("Most Correlative Countries Top 50", fontsize=18)
plt.legend(shadow=True, fontsize=12, loc='lower right', bbox_to_anchor=(.98, 0.05))
fig.autofmt_xdate(rotation=45)

plt.show()

In the UK, happiness and charity seem to be equally correlated.
Chad, Hong Kong, and Congo also seem to have a strong correlation.

Next, let's take a look at the top countries in descending order of their combined happiness and charity scores.

Plot total score¶

sort_by = 'ttl'
df= df_give_happy.sort_values(sort_by, ascending=False)
df = df.iloc[:50]

y_diff = df['ttl']
x = df.index

top = 'Happiness Score'
bot = 'Charities Score'
wit_col = 'with dark color : G20 Member'

fig, ax = plt.subplots(figsize=(18, 6))

plt.bar(x, df['ttl'], label = wit_col, color= 'w', alpha=0.3, width = 0.8)
df_tll = df[['ttl']]

ax_1 = df_tll.plot(kind='bar', alpha=0.5, width = 0.8)
plt.close(ax_1.figure)

for ticks in ax_1.xaxis.get_major_ticks():
    if ticks.label1.get_text() in list(g20):
        ax.patches[df.index.get_indexer([ticks.label1.get_text()])[0]].set_facecolor('gray')

plt.bar(x, df['norm_happiness'], label = top, color= 'goldenrod',alpha=.4, width = 0.8)
plt.bar(x, df['norm_giving'], label = bot, color= 'lightgreen', alpha=.6, width = 0.8,
        bottom = df['norm_happiness'])

plt.xlabel("Total Rank", fontsize=14)
plt.ylabel("Total Score", fontsize=14)
plt.title("Happiness and Charitiable Total Score Rank Top 50", fontsize=18)

handles, labels = plt.gca().get_legend_handles_labels()
order = [0, 2, 1]
plt.legend([handles[idx] for idx in order], [labels[idx] for idx in order],shadow=True, fontsize=12)

fig.autofmt_xdate(rotation=45)

plt.show()

Oceania and North America rank highest.
Most of the top countries are in Europe. In Asia, Singapore is the highest, ranking 15th.
Many of the top countries are characterized by a nearly 1:1 ratio between happiness and charity.

What I would like to focus on here are countries such as Indonesia and Kenya, which do not have a high percentage of happiness, but are ranked because of their high level of charity. These countries have high scores overall.

A question arises here. Are countries with higher GDP happier and more charitable?
Now, let's import GDP data.

GDP Data from wikipedia¶

url = 'https://en.wikipedia.org/wiki/List_of_countries_by_GDP_(nominal)'

df = pd.read_html(url)[2]
df.columns = df.columns.droplevel()

make data frame¶

After merging, the data for Northern Cyprus and Somaliland will missing, so we will add them as new data.

We use the IMF data as the standard, but since it contains missing values, we replace it with the average value between the World Bank and the United Nations.

gd = df.iloc[1:,[0,2,4,6]]
gd.columns = ['country','IMF','WB','UN']

gd = replace_strings_in_dataframe(gd, replacements)
def missing_country_data(df):
    a = df.country.values
    b = df_give_happy.index.values
    return [i for i in b if i not in a]
missing_country_data(gd)
['Northern Cyprus', 'Somaliland Region']
new_data = {'country': ['Northern Cyprus', 'Somaliland Region'],
            'additoinal': [4234, 2500],}
new = pd.DataFrame(new_data)
gd = pd.concat([gd,new], ignore_index=True)

gd = gd.set_index('country').apply(pd.to_numeric, errors='coerce')
gd = gd.T.fillna(gd.T.mean(axis=0), axis=0).T

gd = gd.sort_values('IMF', ascending=False)
pd.options.mode.chained_assignment = None  # default='warn'
gd['gdp'] = range(1, gd.shape[0] +1)
gdp = gd.iloc[:,-1].reset_index()

Merge data with df_give_happy¶

df = df_give_happy.merge(gdp, on='country',how='left')

Normalize¶

df_h = df['gdp']
df['norm_gdp'] = 1 - (df_h - df_h.min()) / (df_h.max() - df_h.min())
gdp.sample(5)

df_t = df['ttl']
df['norm_ttl']= (df_t - df_t.min()) / (df_t.max() - df_t.min())

df_g = df['norm_giving']
df_h = df['norm_happiness']
df['norm_g2'] = df_g * (df['norm_ttl']  / (df_g + df_h))
df['norm_h2'] = df_h * (df['norm_ttl']  / (df_g + df_h))

df_give_happy_gdp = df.copy()

df_give_happy_gdp
country happiness giving norm_happiness norm_giving diff ttl gdp norm_gdp norm_ttl norm_g2 norm_h2
0 Finland 1 74 1.000000 0.493056 0.493056 1.493056 48 0.760204 0.778587 0.257115 0.521472
1 Denmark 2 24 0.993407 0.840278 0.846871 1.833684 38 0.811224 0.958232 0.439106 0.519126
2 Iceland 3 23 0.986813 0.847222 0.860409 1.834035 113 0.428571 0.958417 0.442735 0.515682
3 Israel 4 78 0.980220 0.465278 0.485058 1.445498 29 0.857143 0.753505 0.242539 0.510967
4 Netherlands 5 29 0.973626 0.805556 0.831929 1.779182 17 0.918367 0.929488 0.420842 0.508646
... ... ... ... ... ... ... ... ... ... ... ... ...
162 Somaliland Region 99 145 0.349451 0.000000 0.650549 0.349451 184 0.066327 0.175459 0.000000 0.175459
163 Oman 22 145 0.858242 0.000000 0.141758 0.858242 69 0.653061 0.443792 0.000000 0.443792
164 Djibouti 110 145 0.278022 0.000000 0.721978 0.278022 171 0.132653 0.137788 0.000000 0.137788
165 Guyana 45 145 0.709890 0.000000 0.290110 0.709890 136 0.311224 0.365552 0.000000 0.365552
166 Cuba 69 145 0.551648 0.000000 0.448352 0.551648 68 0.658163 0.282097 0.000000 0.282097

167 rows × 12 columns

color map¶

Outputs a color bar to visually confirm the colors to use for the bar plot.

cmap = plt.get_cmap('spring')

gradient = np.linspace(0, 1, 256).reshape(1, -1)
gradient = np.vstack((gradient, gradient))

fig, ax = plt.subplots(figsize=(20, 1))
ax.set_title('Spring Colormap')
ax.set_xticks([])
ax.set_yticks([])
ax.imshow(gradient, aspect='auto', cmap=cmap)

for i in np.linspace(0, 1, 11):
    position = int(i * 255)
    plt.text(position, 2, f'{i:.1f}', color='black', fontsize=12, ha='center')

plt.show()

Define Plot Function¶

def plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title, begin=0, end=50):
    
    df = df.sort_values(sort_by, ascending=False).set_index('country')
    df = df.iloc[begin:end]

    cmap = plt.get_cmap('spring')
    
    col_g = cmap(0.9) #'yellow'
    col_h = cmap(0.35) #'pink'
    
    top = 'Happiness Score'
    bot = 'Charities Score'
    withg20 = 'with dark color : G20 Member'
    
    shadow = withStroke(linewidth=4, foreground='w')
    
    x = df.index
    df_tll = df[['norm_ttl']]

    fig, ax = plt.subplots(figsize=(18, 6))

    plt.bar(x, df['norm_ttl'], color='gray', alpha=.3, label='with Dark Color: G20 Member')
    plt.bar(x, df['norm_ttl'], label = '', color= 'w', alpha=1, width = 0.8)

    for g in g20:
        if g in x:
            plt.bar(g, df[df.index==g].norm_ttl, color='gray', alpha=0.4)

    plt.bar(x, df['norm_h2'], label = top, color = col_h, alpha=.6, width = 0.8)
    plt.bar(x, df['norm_g2'], label = bot, color = col_g, alpha=.6, width = 0.8,
            bottom = df['norm_h2'])

    avg_norm_ttl = df.norm_ttl.mean()
    plt.axhline(avg_norm_ttl, color='lightskyblue', linestyle='--', label='Average') 
    plt.text(len(df) + 1, avg_norm_ttl,
             f'Avg: {avg_norm_ttl:.2f}',
             va='top', ha='right', color='b').set_path_effects([shadow])
    
    x_position = len(df) / 2 * 1.1
    x_adjust = x_position * 0.2
    
    y_positoin = df['norm_ttl'].max()
    y_adjust = y_positoin * 0.0
    
    plt.text(x_position - x_adjust, y_positoin- y_adjust, 
             f'Avg Charities: {df.norm_g2.mean():.2f}', 
             va = 'bottom',
             ha = 'right',
             color = cmap(0.73),
             fontsize =  12
            ).set_path_effects([shadow])
        
    text_obj = plt.text(x_position + x_adjust, y_positoin-y_adjust,
             f'Avg Happiness: {df.norm_h2.mean():.2f}',
             va = 'bottom',
             ha = 'right',
             color = cmap(.3),
             fontsize = 12
            ).set_path_effects([shadow])

    plt.xlabel(xlabel, fontsize=14)
    plt.ylabel('Total Score', fontsize=14)
    plt.title(title, fontsize=20, pad=18)

    handles, labels = plt.gca().get_legend_handles_labels()
    order = [0, 1, 3, 2]
    
    plt.legend([handles[idx] for idx in order], 
               [labels[idx] for idx in order],
               shadow=True, fontsize=12,
               loc='upper right')

    fig.autofmt_xdate(rotation=45)

    return plt.show()

Plot by GDP Rank¶

df = df_give_happy_gdp
sort_by = 'norm_gdp'
xlabel = 'GDP Rank'
title = 'Happiness and Charitiable Total Score Rank Top 50 sort by GDP'

plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title)

Since the countries are ranked by GDP, the top countries are in the G20.
What I would like to note here is that the height of the bar is not constant. The prediction that economically large countries are happy and benevolent is overturned.

The question of whether economic powers are happy or charitable is that they are not proportionately.

For example, Japan, which is ranked 3rd, has a bar for happiness but no charity, and its total score is about the same as Iraq, which ranks 50th.
Bangladesh and Egypt have very low scores. The average score of the top 50 countries is 0.7.

So what happens below the top 50?

GDP Rank Top 51 to 100¶

df = df_give_happy_gdp
sort_by = 'norm_gdp'
xlabel = 'GDP Rank'
title = 'Happiness and Charitiable Total Score Rank 51 to 100 sort by GDP'

plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title, begin=50, end =100)

New Zealand and Iceland stand out in this graph.

The average score for the top 50 countries was 0.7, but for countries ranked 50th to 100th it was 0.57.
The average for charity is slightly higher than the average for happiness.

Next, let's add the GINI index, which is an inequality index for each country.
There is a table on the Wikipedia page, import it.

Gini Index Rank Data Frame¶

About Gini Coefficient:

In economics, the Gini coefficient, also known as the Gini index or Gini ratio, is a measure of statistical dispersion intended to represent the income inequality, the wealth inequality, or the consumption inequality within a nation or a social group. It was developed by Italian statistician and sociologist Corrado Gini.

List of countries by income equality:

This is a list of countries or dependencies by income inequality metrics, including Gini coefficients. The Gini coefficient is a number between 0 and 1, where 0 corresponds with perfect equality (where everyone has the same income) and 1 corresponds with perfect inequality (where one person has all the income—and everyone else has no income).

Income distribution can vary greatly from wealth distribution in a country (see List of countries by wealth inequality). Income from black market economic activity is not included and is the subject of current economic research.

Load from wikipedia

gini_url = 'https://en.wikipedia.org/wiki/List_of_countries_by_income_equality'
df = pd.read_html(gini_url)[1]
gi = df.droplevel(level=1, axis=1)
gi = gi.iloc[1:-1,[0, -2]]
gi.columns = ['country','rank']
gi = gi.sort_values('rank')

Create data frame¶

Add gini index ranking

gi['gini'] = range(1, len(gi) + 1)
gini = gi[['country','gini']]
replace_strings_in_dataframe(gini, replacements)
country gini
49 Faroe Islands 1
67 Iceland 2
135 Slovakia 3
136 Slovenia 4
38 Czech Republic 5
... ... ...
26 Central African Republic 165
168 Zambia 166
144 Suriname 167
108 Namibia 168
138 South Africa 169

169 rows × 2 columns

Merge with df_give_happy_gdp and Normalize¶

  • Merge Dataframe
  • Missing values will replace with the mean value
  • And normalize gini rank
df = df_give_happy_gdp.merge(gini, on='country', how='left')

df.gini = df.gini.fillna(df.gini.mean())
df_h = df['gini']
df['norm_gini'] = 1 - (df_h - df_h.min()) / (df_h.max() - df_h.min())

df_give_happy_gdp_gini = df.copy()

Sort by lowest Gini index rank, that is, by lowest income disparity.

Plot sort by gini index¶

df = df_give_happy_gdp_gini
sort_by = 'norm_gini'
xlabel = 'Gini Index Rank'
title = 'Happiness and Charitiable Total Score Rank Top 50 sort by Gini Index'

plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title)

And the average total score for happiness and charity came out to be 0.64. This is lower than the GDP ranking.  

The difference between the average level of happiness and the average level of charity was 0.06 points.

Corruption Perceptions Index data¶

Let's also add the Global Corruption Index.

url_cpi = 'https://en.wikipedia.org/wiki/Corruption_Perceptions_Index'

df = pd.read_html(url_cpi)[4]

Create DataFrame¶

cp = df.droplevel(level=0, axis=1)
cp = cp.iloc[0:,[1,0]]
cp.iloc[:,1] =  cp.iloc[:,1].replace('—', np.nan).astype(float)
cp.columns = ['country','cpi']
cp = cp.sort_values('cpi')
replace_strings_in_dataframe(cp, replacements)
country cpi
0 Denmark 1.0
1 New Zealand 2.0
2 Finland 2.0
3 Norway 4.0
4 Singapore 5.0
... ... ...
177 Venezuela 177.0
179 South Sudan 178.0
178 Syria 178.0
180 Somalia 180.0
38 Brunei Darussalam NaN

181 rows × 2 columns

Merge with df_give_happy_gdp_gini¶

df = df_give_happy_gdp_gini.merge(cpi, on='country',how='left')
df.cpi = df.cpi.fillna(df.cpi.mean())

df_h = df['cpi']
df['norm_cpi'] = 1 - (df_h - df_h.min()) / (df_h.max() - df_h.min())

df_give_happy_gdp_gini_cpi = df.copy()

Plot sort by CPI¶

df = df_give_happy_gdp_gini_cpi
sort_by = 'norm_cpi'
xlabel = 'CPI Rank'
title = 'Happiness and Charitiable Total Score Rank Top 50 sort by Corruption Perceptions Index'

plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title)

The average value was 0.72.
The average level of happiness was 0.4, and the average level of charity was 0.32, with the difference increasing to 0.08.
Less corruption seems to make people happier and more charitable, but the increase in happiness appears to be large.

Large economies such as Hong Kong, Japan, and South Korea stand out for their low scores despite having low levels of corruption.

df[:11].norm_ttl.mean()
0.8773524156028142

Surprisingly, the top 11 countries all exceeded the average value of 0.73, and the average was 0.87.


Life Expectancy¶

So what does a country's average life expectancy have to do with happiness and philanthropy? Let's try importing the data.
Wikipedia has tables for the World Bank Group, United Nations, World Health Organization, and CIA World Factbook.

First, combine each into a data frame.

This time we will use data from World Health Organization.

url_life = 'https://en.wikipedia.org/wiki/List_of_countries_by_life_expectancy'
df = pd.read_html(url_life)

Table World Bank¶

le_wb = df[0].droplevel(level=0,axis=1)
le_wb = le_wb.iloc[1:,:2]
le_wb.columns = ['country', 'wb']

le_wb.head()
country wb
1 Hong Kong SAR, China 85.49
2 Macao SAR, China 85.40
3 Japan 84.45
4 Switzerland 83.85
5 South Korea 83.53

Table UN¶

le_un = df[1].droplevel(level=0,axis=1)
le_un = le_un.iloc[1:,:2]
le_un.columns = ['country', 'un']

le_un.head()
country un
1 Hong Kong SAR, China 84.9
2 Japan 84.6
3 Australia 83.4
4 Switzerland 83.8
5 Malta 82.5

Table WHO¶

le_who = df[2].droplevel(level=0, axis=1)
le_who = le_who.iloc[1:,:2]
le_who.columns = ['country','who']

le_who.head()
country who
1 Japan 84.3
2 Switzerland 83.4
3 South Korea 83.3
4 Singapore 83.2
5 Spain 83.2

Table CIA¶

In data of CIA Myanmar called Burma, and West Bank meaning is palestine area.

le_cia = df[3]
le_cia = le_cia.iloc[:227,:2]
le_cia.columns = ['country', 'cia']

le_cia.head()
country cia
0 Monaco 89.52
1 Singapore 86.35
2 Macau 84.98
3 Japan 84.83
4 San Marino 83.86

Replace country name and Merge Each Data Frame¶

le_list = [le_wb, le_who, le_un, le_cia]
for i in le_list:
    replace_strings_in_dataframe(i, replacements)
    
lex = merge_dataframes(le_list)

Check missing country data¶

missing_country_data(lex)
['Northern Cyprus', 'Somaliland Region']

Before combining the data frames, fill the data for Northern cyprus and Somaliland as they are missing.

Data for Somaliland is from UNICEF, and data for Northern Cyprus is from the BBC.

fill_data = {'country': ['Northern Cyprus', 'Somaliland Region'],
            'additional': [(79+83)/2, (55+57)/2],}
fill = pd.DataFrame(fill_data)
lex = pd.concat([lex, fill], ignore_index=True)
lex = lex.set_index('country')
lex['mean'] = lex.T.mean().T
  • Create mean column
  • Fill Nan Value to mean value
  • Create ranking column call lex
lex = lex.reset_index().sort_values('mean',ascending=False)
lex['life_ex'] = range(1, lex.shape[0]+1)
lex= lex.iloc[:,[0,-1]]
lex
country life_ex
207 Monaco 1
1 Macao 2
0 Hong Kong 3
2 Japan 4
6 Singapore 5
... ... ...
201 Chad 236
196 Somalia 237
239 Somaliland Region 238
199 Lesotho 239
198 Central African Republic 240

240 rows × 2 columns

df = df_give_happy_gdp_gini_cpi.merge(lex, on='country', how= 'left')

df_h = df['life_ex']
df['norm_life_ex'] = 1 - (df_h - df_h.min()) / (df_h.max() - df_h.min())

df_give_happy_gdp_gini_cpi_life = df.copy()

Plot sort by Life Expectancy¶

df = df_give_happy_gdp_gini_cpi_life
sort_by = 'norm_life_ex'
xlabel = 'Life Expectancy Rank'
title = 'Happiness and Charitiable Total Score Rank Top 50 sort by Life Expectancy'

plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title)

Monaco ranks first in terms of average life expectancy, followed by Macau, but these two items are not present in the giving index or happieness rank data.

The average value is 0.73, which is a high value.
The average happiness rating was 0.41 and the average charity rating was 0.32, a difference of 0.09.

People seem to be significantly happier as they live longer on average. 

However, low scores stand out in countries with high life expectancy, such as Hong Kong, Japan, and South Korea in top 10 countries.


Natural Disaster Risk¶

Speaking of charity work, one can think of fundraising in times of disaster.
Will donations be made in countries that are prone to natural disasters or surrounding countries, making them more charitable?

This United Nations Office for Disaster Risk Reduction website provides detailed information on natural disaster risks.

Add natural disaster risk data from wikipedia.

url_ndr = 'https://en.wikipedia.org/wiki/List_of_countries_by_natural_disaster_risk'
df = pd.read_html(url_ndr)[0]
ndr = df.iloc[:,[1,0]]
ndr.columns = ['country','ndr']
replace_strings_in_dataframe(ndr, replacements)
country ndr
0 Andorra 1
1 Monaco 1
2 San Marino 2
3 São Tomé and Príncipe 3
4 Luxembourg 4
... ... ...
188 Mexico 181
189 Colombia 182
190 Indonesia 183
191 India 184
192 Philippines 185

193 rows × 2 columns

Merge with df_give_happy_gdp_gini_cpi_life_ndr¶

df = df_give_happy_gdp_gini_cpi_life.merge(ndr, on='country', how='left')
df.ndr = df.ndr.fillna(df.ndr.max())
df_h = df['ndr']
df['norm_ndr'] = 1 - (df_h - df_h.min()) / (df_h.max() - df_h.min())


df_give_happy_gdp_gini_cpi_life_ndr = df.copy()

Plot sort by Natural Disaster Risk¶

df = df_give_happy_gdp_gini_cpi_life_ndr
sort_by = 'norm_ndr'
xlabel = 'Natural Disaster Risk Rank'
title = 'Happiness and Charitiable Total Score Rank Top 50 sort by Natural Disaster (Low) Risk'

plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title)

The mean value was 0.59, and the difference between happiness and benevolence narrowed to just 0.01.
The low risk of natural disasters may have the effect of making benevolence and happiness similar.

Rank 51 to 100¶

df = df_give_happy_gdp_gini_cpi_life_ndr
sort_by = 'norm_ndr'
xlabel = 'Natural Disaster Risk Rank'
title = 'Natural Disaster Risk Ranking 51 to 100'

plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title, 51, 100)

Rank 101 ~¶

df = df_give_happy_gdp_gini_cpi_life_ndr
sort_by = 'norm_ndr'
xlabel = 'Natural Disaster Risk Rank'
title = 'Natural Disaster Risk Ranking 101 to Last'

plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title, 101, -1)

As the risk of natural disasters increased, the gap between benevolence and happiness further decreased.

For those ranked 51st and below, charity was 0.01 points higher than happiness.
For those ranked 101st and below, the level of charity increased to 0.03 points higher than happiness.

Countries that are prone to natural disasters seem to be more benevolent.

Population¶

So what about the impact of population? Import and compare data.

url_pop = 'https://en.wikipedia.org/wiki/List_of_countries_and_dependencies_by_population'
df = pd.read_html(url_pop)

After the integration, Somaliland data will be missing, so we will add it as new data. The numbers are taken from the UNICEF pdf file.

Since China and India account for 34% of the total population, we normalize them by rank rather than population.

po = df[0].iloc[1:,1:3]
po.columns = ['country','num']

new_data = {'country': ['Somaliland Region'], 'num': [4294135]}
new = pd.DataFrame(new_data)
po = pd.concat([po,new], ignore_index=True)

replace_name = {r'\s\(\w+\)': ''}
replace_strings_in_dataframe(po, replace_name)
replace_strings_in_dataframe(po, replacements)
po = po.sort_values('num',ascending=False)
po.insert(1, 'pop', range(1, len(po) + 1))
pop = po.iloc[:,[0,1]]

Merge with df_give_happy_gdp_gini_cpi_life_ndr¶

df = df_give_happy_gdp_gini_cpi_life_ndr.merge(pop, on='country', how='left')

df_h = df['pop']
df['norm_pop'] = 1 - (df_h - df_h.min()) / (df_h.max() - df_h.min())

df_give_happy_gdp_gini_cpi_life_ndr_pop = df.copy()

Plot sort by Population¶

df = df_give_happy_gdp_gini_cpi_life_ndr_pop
sort_by = 'norm_pop'
xlabel = 'Population Rank'
title = 'Happiness and Charitiable Total Score Rank Top 50 sort by Population'

plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title)

The average value is now 0.55. This is the lowest average ever. Interestingly, in the top 50 most populous countries, charity was 0.05 points higher than happiness.

A larger population suggests that people are more charitable.


Suicide Rate¶

In recent years, rising suicide rates have become a major issue in developed countries.

It is known that the incidence of winter depression is higher in northern countries with higher latitudes and shorter daylight hours. Depression is one of the causes of suicide.

How does a country's suicide rate affect happiness and charity?

url_suc = 'https://en.wikipedia.org/wiki/List_of_countries_by_suicide_rate'
df = pd.read_html(url_suc)

Country Data¶

suc_c = df[1].iloc[:183,:2]
suc_c.columns = ['country', 'new']
suc_c.loc[:,'country'] = suc_c['country'].replace(r'\s\*','',regex=True)
suc_c = replace_strings_in_dataframe(suc_c, replacements)
suc_c
country new
0 Afghanistan 6.0
1 Albania 3.7
2 Algeria 2.6
3 Angola 12.6
4 Antigua and Barbuda 0.3
... ... ...
178 Venezuela 2.1
179 Vietnam 7.2
180 Yemen 7.1
181 Zambia 14.4
182 Zimbabwe 23.6

183 rows × 2 columns

Region data¶

suc_r = df[6].iloc[:,[0,3]]
suc_r.columns = ['country', 'old']
suc_r.loc[:,'country'] = suc_r['country'].replace(r'\s\*|\([\w\s]+\)','',regex=True)
suc_r = replace_strings_in_dataframe(suc_r, replacements)

There are two types of data for Taiwan, the government-published value and the estimated value, so we will use the average value.

duplicates = suc_r[suc_r.duplicated(subset='country', keep=False)]
duplicates
country old
9 Taiwan 20.1
17 Taiwan 16.4
merged_suc_r = suc_r.groupby('country')['old'].mean().reset_index()
merged_suc_r[95:100]
country old
95 Switzerland 11.20
96 Syria 0.10
97 São Tomé and Príncipe 0.90
98 Taiwan 18.25
99 Tajikistan 2.60

Merge Country data and Region Data¶

Since there are countries such as Mexico that have values in both dataframes. Add a new column 'Rate' and make sure its content is the value of the 'new' column if it has data, or the value of the 'old' column if the 'new' column is NaN.

suc = suc_c.merge(merged_suc_r, on='country', how='outer')
suc['rate'] = suc['new'].fillna(suc['old'])
suc = suc.iloc[:,[0,-1]]
suc.sort_values('rate')
country rate
186 Saint Kitts and Nevis 0.00
13 Barbados 0.30
4 Antigua and Barbuda 0.30
66 Grenada 0.60
138 Saint Vincent and the Grenadines 1.00
... ... ...
87 Kiribati 30.60
55 Eswatini 40.50
70 Guyana 40.90
184 Greenland 53.34
93 Lesotho 87.50

188 rows × 2 columns

The data shows that Lesotho, which ranks last, stands out.
If we normalize the data as it is, the data will be biased, so we will rank it first.

suc = suc.sort_values('rate')
suc.insert(1, 'suc', range(1, len(suc)+1))
suc = suc.iloc[:,:2]
suc
country suc
186 Saint Kitts and Nevis 1
13 Barbados 2
4 Antigua and Barbuda 3
66 Grenada 4
138 Saint Vincent and the Grenadines 5
... ... ...
87 Kiribati 184
55 Eswatini 185
70 Guyana 186
184 Greenland 187
93 Lesotho 188

188 rows × 2 columns

Merge with df_give_happy_gdp_gini_cpi_life_ndr_pop¶

df = df_give_happy_gdp_gini_cpi_life_ndr_pop.merge(suc, on='country', how='left')
df.suc = df.suc.fillna(df.suc.mean())

df_h = df['suc']
df['norm_suc'] = 1 - (df_h - df_h.min()) / (df_h.max() - df_h.min())

df_give_happy_gdp_gini_cpi_life_ndr_pop_suc = df.copy()

Plot sort by Sucide Rate¶

df = df_give_happy_gdp_gini_cpi_life_ndr_pop_suc
sort_by = 'norm_suc'
xlabel = 'Suicide Rate'
title = 'Happiness and Charitiable Total Score Rank Top 50 sort by Suicide Rate'

plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title)

When the suicide rate is low, the total score is low at 0.54, and benevolence exceeds happiness by 0.02 points.
It doesn't seem to have much to do with happiness.

df = df_give_happy_gdp_gini_cpi_life_ndr_pop_suc
sort_by = 'norm_suc'
xlabel = 'Suicide Rate'
title = 'Happiness and Charitiable Total Score Rank Top 50 sort by Suicide Rate'

plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title, 51,110)

Looking at 50 and below, the total score increased to 0.7 points. This is an unexpected result.


Freedom in the World¶

Add the free index as the last data frame.
Note that this is different from the democracy index.

url_free = 'https://en.wikipedia.org/wiki/Freedom_in_the_World'

df = pd.read_html(url_free) 
w = df[0]
t = df[1]
free_w = w.iloc[1:,[0,5,9,13,17]].droplevel(level=1, axis=1)

free_t = t.iloc[1:,[0,19,23]].droplevel(level=1, axis=1)
free_t.columns = free_t.columns.str.replace(r'\[\d+\]','',regex=True)
fdm = pd.concat([free_w, free_t])# w
fdm = fdm.rename(str.lower, axis='columns')

replace_dic = {
    r'\(.+\).?':'',
    r'.?\*.?':'',
}

replace_strings_in_dataframe(fdm, replace_dic)
replace_strings_in_dataframe(fdm, replacements)
fdm.country = fdm.country.str.strip()
fdm = fdm.set_index('country')

fdm['freedom'] = fdm.T.mean().T

fdm = fdm.sort_values('freedom',ascending=False).iloc[:,-1].reset_index()

Puerto Rico has no freedom world points. However, since this is the United States, enter the same score as your united stats.

pr_row = fdm[fdm["country"] == "Puerto Rico"]
us_row = fdm[fdm["country"] == "United States"]

fdm.loc[pr_row.index, ["freedom"]] = fdm.loc[us_row.index, ["freedom"]].values

fdm[fdm["country"] == "Puerto Rico"]
country freedom
210 Puerto Rico 85.0

Since there are multiple countries with the same score, we will directly normalize the scores without ranking them.

Merge with df_give_happy_gdp_gini_cpi_life_ndr_pop¶

df = df_give_happy_gdp_gini_cpi_life_ndr_pop_suc.merge(fdm, on='country', how='left')

df_h = df['freedom']
df['norm_freedom'] = (df_h - df_h.min()) / (df_h.max() - df_h.min())

df_give_happy_gdp_gini_cpi_life_ndr_pop_suc_freedom = df.copy()

Plot sort by Freedom Rank¶

df = df_give_happy_gdp_gini_cpi_life_ndr_pop_suc_freedom
sort_by = 'norm_freedom'
xlabel = 'Freedom in the World Rank'
title = 'Happiness and Charitiable Total Score Rank Top 50 sort by Freedom Rank'

plot_Happiness_Charitiable_Total(df, sort_by, xlabel, title)

The average score for the top 50 countries was 0.74, the highest ever.
Happiness is 0.42, charity is 0.32, and the gap between them is 0.1 point, which is also the highest difference.

It seems that having a high degree of freedom has a significant effect on happiness. However, there is no effect on charity.


Correlation¶

Let's take a look at the correlation between the features of the data frames added so far.

data = df_give_happy_gdp_gini_cpi_life_ndr_pop_suc_freedom.set_index('country')

cols = [col for col in data.columns if 'norm' in col]

df = data.loc[:,cols[:3]+cols[6:]]

fig = plt.subplots(figsize=(9, 8))
sns.heatmap(df.corr(), annot=True,cmap='PuBu_r')
plt.show()

Conclusion¶

White colors have a stronger correlation, and blue colors have an inverse correlation.

The strongest correlation with happiness is life expectancy, with a correlation rate of 0.79, which is quite strong. It seems that as life expectancy increases, people become happier. The next most correlated factor is the corruption index, which is 0.7. It seems that when corruption decreases, people become happier. Freedom rank also correlates with happiness.

The correlation with natural disaster risk is 0.11, so it seems that even if the risk is low, it does not have much of a relationship with happiness. What is interesting is the population. There is an inverse correlation with happiness at -0.19, indicating that happiness decreases when the population is large.

Happiness is the most correlated factor with Giving. However, the correlation is weak at 0.33. There is also a weak relationship with GDP at 0.3. As people become more financially affluent, philanthropy appears to increase. I speculated that people would become more charitable if there were many natural disasters. In the top 50 countries, it seems that the level of charity has increased, partly because the gap with happiness has narrowed, but at 0.014, there is almost no relationship. And the population bonus seems to have little to do with it. Freedom and life expectancy have very weak correlations with benevolence.

What I would like to draw attention to is the inverse correlation between natural disaster risk and population, which is the most significant -0.49. This may be the result of a tendency to avoid disaster areas as a place of residence. There is also an inverse relationship with GDP, with fewer natural disasters leading to more economic development.

Population appears to have an inverse relationship as a factor. Suicide rates are uncorrelated with almost any factor.

Surprisingly, the results showed a strong relationship between average life expectancy and corruption index correlation are 0.74.

Note about is how to fill in missing values.
GDP, population, and life expectancy are easy to fill in because there is abundant data, but missing values for other items such as the Gini index, corruption index, and natural disaster risk are substituted with average values because of limited sources. . Please also consider that they are not treated as unranked and therefore have some bonuses.


Appendix¶

Let's take a look at the overall rankings of each country. On a scale of 0 to 1, points are 1 for the 1st rank and 0 for the lowest rank.

df= df_give_happy_gdp_gini_cpi_life_ndr_pop_suc_freedom.set_index('country')

cols = [col for col in df.columns[1:] if 'norm' in col]
df_columns = cols[:3]+cols[6:]

df = df.loc[:,df_columns]
df['total'] = df.sum(axis=1)
df = df.sort_values('total')
columns_to_plot = df.columns[:-1]
plt.figure(figsize=(10, 50))

x = df.index
y = df.total

plt.barh(x, y, label = ' ', color= 'gray', alpha=.5)

custom_palette = ["#f5e66b",  # giving
                  "#f482bd",  # happiness
                  "#8de5ec",  # dgp
                  "#e696f0",  # gini
                  "#b39640",  # cpi
                  "#f68189",  # life
                  "#a7c1a9",  # ndr
                  "#96acf0",  # pop
                  "#b3b3b3",  # suc           
                  "#e2fab9",  # free
                 ]  
sns.set_palette(custom_palette)

for i, column in enumerate(columns_to_plot):
    color = sns.color_palette()[i]
    left = df[columns_to_plot[:i]].sum(axis=1)  
    plt.barh(x, df[column], label=' ', color=color, left=left)
    
for j, val in enumerate(df[column]):
    total = y.iloc[j]
    plt.text(total+0.05, j, f'{total:.2f}', va='center',ha='left', color='darkgray')
    
for g in g20:
    if g in x:
        plt.barh(g, df[df.index==g].total, color='gray', alpha=0.4)

plt.plot([], [], ' ', label=" ")

plt.ylim(-1, len(df)-.1)
plt.xlabel('Total Normalized Value',fontsize=14)
plt.ylabel('Country', fontsize=14)
plt.title('Stacked Total Scores Rank of Happiness Index {} countries'.format(len(df)),fontsize=18)

label_list = ["Number is Total Score",
                'with Dark Color: G20 Member',
                'Happiness Index',
                'Giving Index',
                'Gross Domestic Product',
                'Gini Index', 
                'Corruption Perceptions Index',
                'Liefe Expectancy',
                'Natural Disaster Risk',
                'Population',
                'Suicide Rate',
                'Freedom In The World']

L = plt.legend()
for i, l in enumerate(label_list):
    L.get_texts()[i].set_text(label_list[i])

plt.show()

After normalizing each rank, the highest rank of the total score is the Denmark.
The top spots are mostly filled by Europe.
Brazil ranks highest in South America, while Singapore ranks highest in Asia.
The second half of the list is dominated by African countries.

The data contained many missing values, and many countries were not ranked.
Normalization also includes filling in missing values with no data using average values from past ranks.


Thank you!¶

%%html
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.4.2/css/all.min.css">