missesdata - Tumblr blog

missesdata · 2 years ago

Text

Coursera Class "Data Management and Visualization" Peer-graded Assignment 4: Creating Graphs for Your Data

Code:

# -*- coding: utf-8 -*-

import pandas

import numpy

import seaborn

import matplotlib.pyplot as plt

# read in data

data = pandas.read_csv('ool_pds.csv', low_memory=False)

data.head(data)

#make a copy of my data

sub1=data.copy(deep=True)

#make a copy of my data to then just have the relevant variables

sub2 = sub1.copy(deep=True)

sub2 = sub2[[

'W1_C1', 'W1_C1B', 'W1_C2', 'W1_D11', 'W1_D12', 'W1_F1',

'W1_F1A', 'W1_M1', 'W1_M5', 'W1_P2', 'W1_P3', 'PPEDUC',

'PPEDUCAT', 'PPETHM', 'PPGENDER', 'PPINCIMP'

]]

# recode missing values to python missing (NaN)

sub2['W1_C1']=sub2['W1_C1'].replace(-1, numpy.nan)

sub2['W1_F1']=sub2['W1_F1'].replace(-1, numpy.nan)

sub2['W1_M5']=sub2['W1_M5'].replace(-1, numpy.nan)

sub2['W1_M5']=sub2['W1_M5'].replace(7, numpy.nan)

sub2['W1_M5']=sub2['W1_M5'].replace(8, numpy.nan)

#recode "attendance of religious service" so that the higher number means "more"

recode1 = {1: 5, 2: 4, 3: 3, 4: 2, 5: 1, 6: 0}

sub2['W1_M5_rec']= sub2['W1_M5'].map(recode1)

#recode attendance of religious service into a quasi-continuous variable

#attend religious service per year

recode2 = {0: 0, 1: 1, 2: 6, 3: 18, 4: 52, 5: 73}

sub2['W1_M5_cont']= sub2['W1_M5_rec'].map(recode2)

#univariate bar graph for categorical variables

# First change format from numeric to categorical

sub2["W1_C1"] = sub2["W1_C1"].astype('category')

seaborn.countplot(x="W1_C1", data=sub2)

plt.xlabel('1=Republican; 2=Democrat; 3=Independent; 4= Other')

plt.title('Counts for political preference in the Outlook on Life survey')

sub2["W1_F1"] = sub2["W1_F1"].astype('category')

seaborn.countplot(x="W1_F1", data=sub2)

plt.xlabel('1=Optimist; 2=Neither; 3=Pessimist')

plt.title('Counts for optimists in the Outlook on Life survey')

sub2["W1_M5_rec"] = sub2["W1_M5_rec"].astype('category')

seaborn.countplot(x="W1_M5_rec", data=sub2)

plt.xlabel('0=Never; 1=Once a year or less; 2=A few times a year; 3= Once or twice a month; 4=Once a week; 5=More than once a week')

plt.title('Counts for attendance of religious services in the Outlook on Life survey')

#hypothesis "test" via graphical inspection

#second way: look at optimism and religiousity as 0/1

#Create a variable "Being an optimist" as 0=no/1=yes from the W1_F1 optimism.

recode4 = {1:1, 2:0, 3:0}

sub2['W1_F1_optimist']= sub2['W1_F1'].map(recode4)

sub2["W1_F1_optimist"] = sub2["W1_F1_optimist"].astype('category')

seaborn.countplot(x="W1_F1_optimist", data=sub2)

plt.xlabel('0=Other; 1=Optimist')

plt.title('Optimism yes=1/no=0 in the Outlook on Life survey')

#let's see if optimists are rather conservatives

#set W1_F1_optimist to numeric so that the proportion (=a Q variable) can be shown on the y axis of the bar chart

sub2["W1_F1_optimist"] = pandas.to_numeric(sub2["W1_F1_optimist"])

# bivariate bar graph C->Q

seaborn.catplot(x="W1_C1", y="W1_F1_optimist", data=sub2, kind="bar", ci=None)

plt.xlabel('Pol Party Pref: 1=Republican; 2=Democrat; 3=Independent; 4= Other')

plt.ylabel('Proportion Optimists')

#let's see if more religious people are rather conservatives

#Create a variable "Being religious" as 0=no/1=yes from the W1_M5_rec attendance of religious services.

recode5 = {0:0, 1:1, 2:1, 3:1, 4:1, 5:1}

sub2['W1_M5_religious']= sub2['W1_M5_rec'].map(recode5)

sub2["W1_M5_religious"] = sub2["W1_M5_religious"].astype('category')

seaborn.countplot(x="W1_M5_religious", data=sub2)

plt.xlabel('0= not religious; 1=religious')

plt.title('Attendance of religious services yes=1/no=0 in the Outlook on Life survey')

#set W1_M5_religious to numeric so that the proportion (=a Q variable) can be shown on the y axis of the bar chart

sub2["W1_M5_religious"] = pandas.to_numeric(sub2["W1_M5_religious"])

# bivariate bar graph C->Q

seaborn.catplot(x="W1_C1", y="W1_M5_religious", data=sub2, kind="bar", ci=None)

plt.xlabel('Pol Party Pref: 1=Republican; 2=Democrat; 3=Independent; 4= Other')

plt.ylabel('Proportion religious')

Output and explanation:

This graph shows the distribution of political preference in the Outlook on Life survey. The majority of respondents prefer the democratic party, followed by independents, repbulicans and others. As this this a nominal variable and no ordered, interval or ratio scale, the right skew that seems to show here is not interpretable. The same goes for the other nominal variables below.

This graph shows the distribution of optimism in the Outlook on Life survey. The majority of respondents report to be optimists, followed by neither and pessimists.

Most attends religious services once a week, followed by "never" and "a few times a year". Still fewer attend once a year or several times a week and the smallest group attends once or twice a month.

Data were grouped into a new variable with opimists vs. the rest to have a categorical variable with just two categories (yes/no). That way, the proportion of people with certain political preferences can be shown in a bar chart.

Data were grouped into a new variable with people attending relicious services vs. the rest to have a categorical variable with just two categories (yes/no). That way, the proportion of people with certain political preferences can be shown in a bar chart.

Among those who self-report to be republicans, about 53% also self-report to be optimists, and among those who self-report to be democrats, about 58% self-report to be optimists.

Among those who self-report to be republicans, about 88% also self-report to attend religious services, and among those who self-report to be democrats, about 82% self-report to attend religious services.

#coursera

0 notes

missesdata · 2 years ago

Text

Coursera Class "Data Management and Visualization" Peer-graded Assignment 3: Making data management decisions

******************************************

Program:

******************************************

# import libraries

import pandas

import numpy

# read in data

data = pandas.read_csv('ool_pds.csv', low_memory=False)

#make a copy of my data

sub1=data.copy(deep=True)

#make a copy of my data to then just have the relevant variables

sub2 = sub1.copy(deep=True)

sub2 = sub2[[

'W1_C1', 'W1_C1B', 'W1_C2', 'W1_D11', 'W1_D12', 'W1_F1',

'W1_F1A', 'W1_M1', 'W1_M5', 'W1_P2', 'W1_P3', 'PPEDUC',

'PPEDUCAT', 'PPETHM', 'PPGENDER', 'PPINCIMP'

]]

#number of observations (rows)

print ("# of cases:", len(sub2))

# number of variables (columns)

print ("# of variables:", len(sub2.columns))

print ("-----------------------------")

#print the counts of the original variable

print ('counts for political preference (original W1_C1)')

print ('1=Republican; 2=Democrat; 3=Independent; 4= Other; -1=no answer')

c1 = sub2['W1_C1'].value_counts(sort=False, dropna=False)

print(c1)

print ("-----------------------------")

# recode missing values to python missing (NaN)

sub2['W1_C1']=sub2['W1_C1'].replace(-1, numpy.nan)

#print the counts of the recoded variable

print ('counts for political preference (recoded W1_C1)')

print ('1=Republican; 2=Democrat; 3=Independent; 4= Other; NaN=missing value')

c1 = sub2['W1_C1'].value_counts(sort=False, dropna=False)

print(c1)

print ("-----------------------------")

#print the counts of the original variable

print ('Counts for being an optimist/pessimist (original W1_F1)')

print ('1=Optimist; 2=Neither; 3=Pessimist; -1=no answer')

c2 = sub2['W1_F1'].value_counts(sort=False, dropna=False)

print(c2)

print ("-----------------------------")

# recode missing values to python missing (NaN)

sub2['W1_F1']=sub2['W1_F1'].replace(-1, numpy.nan)

#print the counts of the recoded variable

print ('Counts for being an optimist/pessimist (recoded W1_F1)')

print ('1=Optimist; 2=Neither; 3=Pessimist; NaN=missing value')

c2 = sub2['W1_F1'].value_counts(sort=False, dropna=False)

print(c2)

print ("-----------------------------")

#print the counts of the original variable

print ('Counts for attendance of a religious service (original W1_M5)')

print ('1=More than once a week; 2=Once a week; 3=Once or twice a month')

print ('4=A few times a year; 5=Once a year or less; 6=Never; 7=Not asked')

print ('8=Missing; -1=no answer')

c3 = sub2['W1_M5'].value_counts(sort=False, dropna=False)

print(c3)

print ("-----------------------------")

# recode missing values to python missing (NaN)

sub2['W1_M5']=sub2['W1_M5'].replace(-1, numpy.nan)

sub2['W1_M5']=sub2['W1_M5'].replace(7, numpy.nan)

sub2['W1_M5']=sub2['W1_M5'].replace(8, numpy.nan)

#print the counts of the recoded variable

print ('Counts for attendance of a religious service (recoded W1_M5)')

print ('1=More than once a week; 2=Once a week; 3=Once or twice a month')

print ('4=A few times a year; 5=Once a year or less; 6=Never; NaN=missing value')

c3 = sub2['W1_M5'].value_counts(sort=False, dropna=False)

print(c3)

print ("-----------------------------")

#recode "attendance of religious service" so that the higher number means "more"

recode1 = {1: 5, 2: 4, 3: 3, 4: 2, 5: 1, 6: 0}

sub2['W1_M5_rec']= sub2['W1_M5'].map(recode1)

#print the counts of the recoded variable

print ('Counts for attendance of a religious service (recoded W1_M5)')

print ('5=More than once a week; 4=Once a week; 3=Once or twice a month')

print ('2=A few times a year; 1=Once a year or less; 0=Never; NaN=missing value')

c4 = sub2['W1_M5_rec'].value_counts(sort=False, dropna=False)

print(c4)

print ("-----------------------------")

#recode attendance of religious service into a quasi-continuous variable

#attend religious service per year

recode2 = {0: 0, 1: 1, 2: 6, 3: 18, 4: 52, 5: 73}

sub2['W1_M5_cont']= sub2['W1_M5_rec'].map(recode2)

#print the counts of the recoded variable

print ('counts for attendance of a religious service')

print ('as number per year (recoded W1_C1_cont)')

c5 = sub2['W1_M5_cont'].value_counts(sort=False, dropna=False)

print(c5)

print ("-----------------------------")

**************************************

output

**************************************

# of cases: 2294

# of variables: 18

-----------------------------

counts for political preference (original W1_C1)

1=Republican; 2=Democrat; 3=Independent; 4= Other; -1=no answer

1.0 331

2.0 1251

3.0 555

4.0 108

NaN 49

Name: W1_C1, dtype: int64

-----------------------------

counts for political preference (recoded W1_C1)

1=Republican; 2=Democrat; 3=Independent; 4= Other; NaN=missing value

1.0 331

2.0 1251

3.0 555

4.0 108

NaN 49

Name: W1_C1, dtype: int64

-----------------------------

Counts for being an optimist/pessimist (original W1_F1)

1=Optimist; 2=Neither; 3=Pessimist; -1=no answer

1.0 1225

2.0 796

3.0 218

NaN 55

Name: W1_F1, dtype: int64

-----------------------------

Counts for being an optimist/pessimist (recoded W1_F1)

1=Optimist; 2=Neither; 3=Pessimist; NaN=missing value

1.0 1225

2.0 796

3.0 218

NaN 55

Name: W1_F1, dtype: int64

-----------------------------

Counts for attendance of a religious service (original W1_M5)

1=More than once a week; 2=Once a week; 3=Once or twice a month

4=A few times a year; 5=Once a year or less; 6=Never; 7=Not asked

8=Missing; -1=no answer

1.0 356

2.0 534

NaN 32

4.0 405

5.0 326

6.0 407

3.0 234

Name: W1_M5, dtype: int64

-----------------------------

Counts for attendance of a religious service (recoded W1_M5)

1=More than once a week; 2=Once a week; 3=Once or twice a month

4=A few times a year; 5=Once a year or less; 6=Never; NaN=missing value

1.0 356

2.0 534

NaN 32

4.0 405

5.0 326

6.0 407

3.0 234

Name: W1_M5, dtype: int64

-----------------------------

Counts for attendance of a religious service (recoded W1_M5)

5=More than once a week; 4=Once a week; 3=Once or twice a month

2=A few times a year; 1=Once a year or less; 0=Never; NaN=missing value

5.0 356

4.0 534

NaN 32

2.0 405

1.0 326

0.0 407

3.0 234

Name: W1_M5_rec, dtype: int64

-----------------------------

counts for attendance of a religious service

as number per year (recoded W1_C1_cont)

73.0 356

52.0 534

NaN 32

6.0 405

1.0 326

0.0 407

18.0 234

Name: W1_M5_cont, dtype: int64

-----------------------------

*************************************

Explanation

*************************************

The Outlook on Life Surveys were conducted by the GfK Knowledge Networks and study political and social attitudes in the United States. A total of 2,294 respondents participated in this study during Wave 1. The sample consisted of African American/Black males and females and White/other race males and females. All participants were aged 18 older and all were non-institutionalized and residing in the United States.

Three questions were looked at here:

Question 1:

"Generally speaking, do you usually think of yourself as a Democrat, a Republican, an Independent, or something else?" (W1_C1)

A little over half of the respondents identified as democrats (n=1251 or 54.5%), one fourth as independent (n=555 or 24.2%), 14.4% (n=331) as republican and 4.7% (n=108) as favoring something else. From 49 people (2.1%) people no answers were collected for this question.

Missing values were recoded into "NaN".

Question 2:

"When you think about your future, are you generally optimistic, pessimistic, or neither optimistic nor pessimistic?" (W1_F1)

A little over half of the respondents identified as optimists (n=1225 or 53.4%), a little over a third as neither opitimist nor pessimist (n=796 or 34.7%) and close to one tenth (n=218 or 9.5%) as pessimist. From 55 people (2.4%) no answers were collected for this question.

Missing values were recoded into "NaN".

Question 3:

"How often do you attend religious services?" (W1_M5)

Almost one fourth (n=534 or 23.3%) attends religious services once a week, 356 people (15.4%) attend more than once a week, 234 people (10.2%) attend once or twice a month, 405 people (17.7%) attend a few times a year, 326 people (14.2%) attend once a year or less and 407 poeple (17.7%) never attend any religious services. From 17 people (0.7%) no data were collected although the question was asked and 15 people (0.7%) were not asked that question.

First, missing values were recoded into "NaN".

Second, the variable was recoded so that low codes would correspond with fewer visits of religious services.

Third, a the variable was recoded into a variable containing the number of religious services attended per year.

Here, we see that 407 people do not attend any services, 326 people attend one service per year, 405 people attend about 6 services per year, 234 attend about 18 services per year, 534 people attend 52 services per year, and 356 people attend an estimated number of 73 services per year.

Note, however, that there is a considerable amout of guesswork involved in this last recoding and this recoded variable may not be suitable for further analyses.

#coursera

0 notes

missesdata · 2 years ago

Text

Coursera Class "Data Management and Visualization" Peer-graded Assignment 2: Running your first program

************************************************************************

Program:

************************************************************************

# import libraries

import pandas

import numpy

# read in data

data = pandas.read_csv('ool_pds.csv', low_memory=False)

data.head(data)

#number of observations (rows)

print ("# of cases:", len(data))

# number of variables (columns)

print ("# of variables:", len(data.columns))

print ("-----------------------------")

# print the counts

print ('Counts for political preference')

print ('1=Republican; 2=Democrat; 3=Independent; 4= Other; -1=no answer')

w1_c1 = data['W1_C1'].value_counts(sort=False)

print (w1_c1)

print ("-----------------------------")

#print the percentages as fraction from 1

print ('Percentages for political preference')

print ('1=Republican; 2=Democrat; 3=Independent; 4= Other; -1=no answer')

p_w1_c1 = data['W1_C1'].value_counts(sort=False, normalize=True)

print (p_w1_c1)

print ("-----------------------------")

# print the counts

print ('Counts for being an optimist/pessimist')

print ('1=Optimist; 2=Neither; 3=Pessimist; -1=no answer')

w1_f1 = data['W1_F1'].value_counts(sort=False)

print (w1_f1)

print ("-----------------------------")

#print the percentages as fraction from 1

print ('Percentages for being an optimist/pessimist')

print ('1=Optimist; 2=Neither; 3=Pessimist; -1=no answer')

p_w1_f1 = data['W1_F1'].value_counts(sort=False, normalize=True)

print (p_w1_f1)

print ("-----------------------------")

# print the counts

print ('Counts for attendance of a religious service')

print ('1=More than once a week; 2=Once a week; 3=Once or twice a month')

print ('4=A few times a year; 5=Once a year or less; 6=Never; 7=Not asked')

print ('8=Missing; -1=no answer')

w1_m5 = data['W1_M5'].value_counts(sort=False)

print (w1_m5)

print ("-----------------------------")

#print the percentages as fraction from 1

print ('Percentages for attendance of a religious service')

print ('1=More than once a week; 2=Once a week; 3=Once or twice a month')

print ('4=A few times a year; 5=Once a year or less; 6=Never; 7=Not asked')

print ('8=Missing; -1=no answer')

p_w1_m5 = data['W1_M5'].value_counts(sort=False, normalize=True)

print (p_w1_m5)

************************************************************************Output

************************************************************************

# of cases: 2294

# of variables: 436

-----------------------------

Counts for political preference

1=Republican; 2=Democrat; 3=Independent; 4= Other; -1=no answer

2 1251

4 108

-1 49

1 331

3 555

Name: W1_C1, dtype: int64

-----------------------------

Percentages for political preference

1=Republican; 2=Democrat; 3=Independent; 4= Other; -1=no answer

2 0.545336

4 0.047079

-1 0.021360

1 0.144289

3 0.241935

Name: W1_C1, dtype: float64

-----------------------------

Counts for being an optimist/pessimist

1=Optimist; 2=Neither; 3=Pessimist; -1=no answer

2 796

-1 55

1 1225

3 218

Name: W1_F1, dtype: int64

-----------------------------

Percentages for being an optimist/pessimist

1=Optimist; 2=Neither; 3=Pessimist; -1=no answer

2 0.346992

-1 0.023976

1 0.534002

3 0.095031

Name: W1_F1, dtype: float64

-----------------------------

Counts for attendance of a religious service

1=More than once a week; 2=Once a week; 3=Once or twice a month

4=A few times a year; 5=Once a year or less; 6=Never; 7=Not asked

8=Missing; -1=no answer

2 534

4 405

6 407

-1 17

1 356

3 234

5 326

7 15

Name: W1_M5, dtype: int64

-----------------------------

Percentages for attendance of a religious service

1=More than once a week; 2=Once a week; 3=Once or twice a month

4=A few times a year; 5=Once a year or less; 6=Never; 7=Not asked

8=Missing; -1=no answer

2 0.232781

4 0.176548

6 0.177419

-1 0.007411

1 0.155187

3 0.102005

5 0.142110

7 0.006539

Name: W1_M5, dtype: float64

************************************************************************

Description

************************************************************************

Three questions were looked at here:

Question 1:

"Generally speaking, do you usually think of yourself as a Democrat, a Republican, an Independent, or something else?" (W1_C1)

Question 2:

"When you think about your future, are you generally optimistic, pessimistic, or neither optimistic nor pessimistic?" (W1_F1)

Question 3:

"How often do you attend religious services?" (W1_M5)

#coursera

0 notes

missesdata · 2 years ago

Text

Coursera Class "Data Management and Visualization" Peer-graded Assignment: Getting Your Research Project Started

STEP 1 - Review the codebooks

The codebook of the Outlook on Life Survey is the most appealing and intuitive to me. I am interested in the associations between various personal characteristics and polititcal opinion, e.g., preference for or rating of a political party in the U.S.A.

STEP 2 - Identify my topic of interest

Specifically, I am interested in the reasons for party preference. How do the supporters of the policital parties differ in terms of their personaliy? Are certain personal characteristics related to the preference for a political party (in the U.S.A.)?

More specific:

Are optimistic people rather republicans or democrats? Are religious people rather republicans or democrats?

STEP 3 - Select the relevant variables from the codebook and prepare your own codebook*

p. 26: W1_C1: Generally speaking, do you usually think of yourself as a Democrat, a Republican, an Independent, or something else?

p. 29: W1_C2: We hear a lot of talk these days about liberals and conservatives. Where would you place YOURSELF on this 7 point scale?

p. 79: W1_F1: When you think about your future, are you generally optimistic, pessimistic, or neither optimistic nor pessimistic?

p. 115: W1_M5: How often do you attend religious services?

Other related variables or variable subsitutes in codebook:

p. 28: W1_C1B: Would you call yourself a strong [Democrat/Republican] or a not very strong [Democrat/Republican]?

p. 46: W1_D11: [The Republican Party] How would you rate:

p. 48: W1_D12: [The Democratic Party] How would you rate:

p. 113: W1_M1: What is your religion?

*I prepared my own codebook, but I have not included it here because it is quite lengthy.

STEP 4 - Identify a second topic of interest

Besides personal characteristics, there may be also more situational characteristics like social class, income, educatiom or neighborhood. Therefore, I am also interested in probing for those. Are socio-demographic characteristics associated with the preference for a political party (in the U.S.A.)?

More specific:

Are people with higher income rather republicans or democrats? Are people with a higher educational degree rather republicans or democrats?

STEP 5 - Select the relevant variables from the codebook and prepare your own codebook*

p. 263: PPINCIMP: Household Income

p. 260: PPEDUC: Education (Highest Degree Received)

Other related variables in codebook:

p. 146: W1_P2: People talk about social classes such as the poor, the working class, the middle class, the upper-middle class, and the upper class. Which of these classes would you say you belong to?

p. 146: W1_P3: Which of these classes would you say most memebers of your family belong to?

*I prepared my own codebook, but I have not included it here because it is quite lengthy.

STEP 6 - Literature review

Search terms for literature research in google.com and scholar.google.com:

optimism and party preference USA

optimism and political affiliation US

religiosity and republicans

religiosity and democrats

republicans democrats income

republicans democrats education

republicans democrats social class

References:

Ozmen, C.B., Brelsford, G.M. & Danieu, C.R. Political Affiliation, Spirituality, and Religiosity: Links to Emerging Adults’ Life Satisfaction and Optimism. J Relig Health 57, 622–635 (2018). https://doi.org/10.1007/s10943-017-0477-y

Found that politically conservatives were more optimistic

Found that republicans were more religious

Claassen, R.L. (2015). Godless Democrats and Pious Republicans?: Party Activists, Party Capture, and the 'God Gap'. Cambridge University Press, Cambridge.

Discusses why there is the conviction that republicans are more religious than democrats

Hanson, P. & Chen, Y. (2020). The demographic profiles of democrats and republicans. Data Analysis and Social Inquiry Lab, Grinnell College. https://dasil.sites.grinnell.edu/2020/05/the-demographic-profiles-of-democrats-and-republicans/

Found that republicans earn less than democrats

Found that democrats are more educated than republicans

Ahler DJ, Sood G. 2018. The parties in our heads: misperceptions about party composition and their consequences. J. Politics 80(3):964–81. http://www.dougahler.com/uploads/2/4/6/9/24697799/ahlersood_pioh_forthcoming_jop.pdf

Found that people overestimate the number of "rich" people (earning more than 250,00 $ per year) in the republican party: estimation: 38.2% vs. real number: 2.2%; Republicans are not as rich as common sense might suggest.

STEP 7 - Hypotheses based on literature review

H1: People who self-identify as conservative/republican are more optimistic. (W1_C1 / W1_C2; W1_F1)

H2: People who self-identify as conservative/republican more often report to be strongly religious. (W1_C1 / W1_C2; W1_M5)

H3: People who self-identify as conservative/republican have a lower income. (W1_C1 / W1_C2; PPINCIMP)

H4: People who self-identify as conservative/republican have a lower education level. (W1_C1 / W1_C2; PPEDUC)

#coursera

0 notes