Analysis

Documentation of the code in experiment_floodplain/analysis. The analysis consists of three tasks:

• Task 1: In the folder task_descriptive_analysis. These scripts compute summary statistics over survey respondents’ beliefs (task_summary_beliefs.py), information quality (task_summary_information_frictions.py) and reading behavior during the experimental part of the survey (task_summary_reading_behavior.py).

• Task 2: In the folder task_check_experiment_randomization. Checks for the integrity of the experimental randomization (task_check_randomization.py) and compare characteristics of survey respondents, survey recipients, and full sample (task_check_survey_respondents.py).

• Task 3: In the folder task_estimate_treatment_effects.

Scripts for Task 1: descriptive analysis

Script task_summary_beliefs.py

This script compute summary statistics for survey respondents’ baseline beliefs. The results are saved as .csv files in bld/analysis/beliefs. These are:

Summary statistics over reported beliefs

File name	Summary statistics of …
summary_beliefs_all.csv	All prior beliefs (10-year flood probability, damages, total and government compensation)
summary_beliefs_quartiles.csv	All prior beliefs (mean, median, and standard deviation) by quartile.
summary_beliefs_risk.csv	Prior beliefs about 10-year flood probability, by reported and true flood risk categories.
summary_beliefs_damages.csv	Prior beliefs about damages, by reported and true maximum water depth categories.
summary_beliefs_t1.csv	Prior beliefs, posterior beliefs, and belief updates under treatment 1 of the survey experiment (“neutral text”).
summary_beliefs_updates.csv	All belief updates.
summary_beliefs_updates_direction.csv	Belief updates, by expected direction of the updates (based on baseline information frictions).
summary_risk_updates_direction.csv	Belief update over 10-year flood probability, by expected direction of the updates (based on baseline information frictions), tabulated by objective flood risk category.

Script task_summary_information_frictions.py

This script compute summary statistics for survey respondents’ answers to information-based questions in the survey. The results are saved as .csv files in bld/analysis/information. These are:

Summary statistics over information quality

File name	Summary statistics, questions on…
frictions_maps.csv	Flood maps
frictions_insurance.csv	Insurance availability.
frictions_claims.csv	Claims paid out by the government in July 2021.
frictions_wts.csv	WTS act on government compensation.
confidently_incorrect_beliefs.csv	Confidence in answers to information-based questions, people who are confidently incorrect vs. people who are not.

Script task_summary_reading_behavior.py

This script computes summary statistics on the reading behavior of survey respondents during the experimental part of the survey. The results are saved as .csv files in bld/analysis/information.

Summary statistics over reading behavior

File name	Summary statistics for …
reading_behavior.csv	Time spent reading.
read_attention.csv	Reading behavior conditional on people’s attention to the experimental text, proxied by text comprehension question.
read_nothing.csv	Characteristics of people who read nothing besides the experimental text.
clicks_vs_frictions.csv	Results of Logit models for relationship between survey respondents’ information frictions by topic and whether they click on the corresponding text.

Scripts for Task 2: randomization checks

Script task_check_randomization.py

This script checks whether the randomization worked as expected, for a range of pre-specified covariates stored as covariates_randomization.csv in analysis/csv. The prespecified covariates are listed in the pre-analysis plan (Section IV, Analysis, sub-section C, Treatment assignment, paragraph Integrity of randomization).

The results are saved as randomization.csv in bld/analysis/randomization.

Script task_check_survey_respondents.py

This script compares survey recipients to survey respondents and the full sample of eligible households, for a range of covariates stored in covariates_respondents.csv in analysis/csv. The results are saved as samples_covariates.csv in bld/analysis/covariates.

Scripts for Task 3: analysis of experimental results

Script task_estimate_ATE.py

This script estimates the Average Treatment Effects of the intervention on the (pre-specified) measures of beliefs updating and worry about flood risk, as well as the (pre-specified) measures of willigness-to-pay for insurance (hypothetical) and for information about Dutch insurance companies that offer protection against flood risk (incentivized).

I use (1) an unadjusted linear regression estimated via OLS (outcomes_unadjusted.csv), (2) a linear regression estimated via OLS that includes a small set of pre-specified covariates (outcomes_precovs.csv), and (3) two partially linear models that include a broader set of covariates where the nuisance functions are estimated via Lasso (outcomes_rlasso_post.csv and outcomes_rlasso_double.csv). I adjust the p-values for multiple hypotheses testing using the two-stage Benjamini, Krieger, and Yekutieli procedure for controlling the false discovery rate (FDR) as implemented in the Python package statsmodels (option “fdr_tbsky”). The four dataframes of results are saved as .csv files in bld/analysis/outcomes.

I additionally test whether survey respondents have a different probability to answer any given outcome-related question across treatment arms, via a logistic regression where the dependent variable indicates whether such outcome is present in the data. The results for it are saved as whether_outcome_present.csv, also in bld/analysis/outcomes.

Script task_run_rlasso.py

This script runs the R script run_lasso.R in experiment_floodplain/analysis/R and saves the result as .csv files to bld/analysis.

The the R script run_lasso.R uses the package hdm (High-Dimensional Metrics, [1]) to select controls for the ATE estimationg via Lasso end Post-Lasso methods for for high-dimensional approximately sparse models. The set of all controls is in analysis/csv/formulas (all the variables with VARTYPE equal to PRE_COV). Each .csv file contains the results of the estimation for a different (pre-specified) dependent variable. The .csv files are used by the script task_estimate_ATE.py in a later step.

Script task_heterogeneity_analysis.py

This script estimates heterogeneous treatment effects for a few pre-specified variables. The dataframes of results are saved as .csv files in bld/analysis/heterogeneity. In each file, each row represents an independent variable, while each column represents a dependent variable. These files are:

Heterogeneous treatment effects

File name	Results for interaction with…
interaction_belief_confidence.csv	Confidence in prior beliefs (standardized), for each belief.
interaction_prior_beliefs.csv	Prior beliefs, for each belief.
interaction_confidently_wrong.csv	Indicator for respondents who give at least one confidently wrong answer. plus the two willingness-to-pay measures.
interaction_total_frictions.csv	Variable counting the number of incorrect answer in the survey section on information-based questions.
interaction_frictions_by_topic.csv	Indicator for information friction in a certain topic (flood maps, government compensation, insurance).
interaction_belief_update.csv	Belief update, for each belief.
interaction_belief_update_any.csv	Indicator for whether the respondent update their beliefs, for each belief.
interaction_flood_experience.csv	Indicator for self-reported experience of flood damages in 2021.

Auxiliary modules

Contain functions used in the scripts listed above.

Script descriptive_stats.py

In the folder PYTHON.

Module to compute descriptive statistics (first step of the analysis).

assess_risk_update_direction(df)

Create column indicating whether belief updating over 10-year flood probability is expected or unexpected based on baseline information frictions with respect to Risicokaart flood maps.

compute_mean_and_ttest(df, var, split_by)

Split data under column var of df by the binary variable split_by, compute the mean of the two groups and test for their differrence (t-test, the variance of the two samples is not assumed to be equal).

compute_share_by_columns(df)

Replace values of each df column with its share with respect to the column total. Add row with total.

compute_share_of_updates(df, belief)

Compute share of belief update, for expected updates given baseline information frictions.

Parameters:

• df (Pandas.DataFrame) – Dataframe.

• belief (str) – Belief.

Returns:

Pandas.DataFrame.

compute_stats_by_treatment(df, columns)

Get mean, median, and standard deviation of columns in Pandas.DataFrame df.

compute_updating_stats(df, belief)

Compute share of belief updates, share of updates by direction, and average update size by direction conditional on treatment.

get_conditional_summary_stats(df, belief, col, vals, colnames)

Compute summary statistics of column belief in df, conditional on col taking each value in vals list.

Summary statistics are: minimum value, 25th quantile, mean, median, 75th quantile, maximum value, standard deviation (SD) and Mean Absolute Deviation (MAD).

Parameters:

• df (Pandas.DataFrame) – DataFrame containing column belief.

• belief (str) – Name of column of which summary statistics should be computed.

• col (str) – Name of column conditional on which summary statistics for belief should be computed.

• vals (list) – Values taken by col.

• colnames (list of str) – Name of columns in DataFrame of results (one name for each value in vals).

Returns:

Pandas.DataFrame.

get_covariates_dataset(covariatesDF, valuesDF, weights=False)

Compute (weighted) average values of variables in covariatesDF from valuesDF. Missing values are automatically excluded.

Parameters:

• covariatesDF (pandas.DataFrame) – Dataframe of covariates. Need to have columns named “CATEGORY”, “DESCRIPTION”, “VARNAME”, and “WEIGHTS”.

• valuesDF (pandas.DataFrame) – Dataframes of covariates’ values. Need to have a column named “VARNAME” containing the variables in covariatesDF.

• weights (Bool) – Weights to compute weighted mean. If True, will be pulled from valuesDF.WEIGHTS. Default is False

Returns:

pandas.DataFrame

get_maps_frictions(df, mapinfo, mapdict)

Compare correct vs. stated information based on flood maps.

get_spearman_rho(df, varlist)

Compute Spearman’s rho for df columns in varlist.

Parameters:

• df (Pandas.DataFrame) – Must contains columns in varlist.

• varlist (2-dimensional list) – List of paired names of columns the Spearman’s rho should be computed for, e.g.: [[“A”, “B”], [“C”, “D”]] will return association coefficient between A and B, and C and D.

Returns:

List of floats.

get_summary_stats(df, beliefs, colnames=False)

Get summary statistics for beliefs columns in df.

Summary statistics are: minimum value, 25th quantile, mean, median, 75th quantile, maximum value, standard deviation (SD) and Mean Absolute Deviation (MAD).

make_conditional_belief_updates_table(df, groupby_col, groupby_col_name)

Tabulate direction of belief updates over 10-year flood probability, classified based on baseline information frictions with respect to Risicokaart flood maps (return period of a flood at the address of residence), conditional on column groupby_col.

make_table_adminlevel_residents(gdf, adminLevels)

Compute number of residents in gdf dataset for each administrative level in adminLevels.

make_table_balance(df_names, dfs, col_dict)

Compute average values in dfs for columns specified in col_dict dictionary, as well as t-statistic and p-values for the mean difference of the two distributions.

Parameters:

• df_names (list of str) – Name of two dataframes in dfs.

• dfs (list of pandas.Dataframes) – List of two dataframes.

• col_dict (dict) – Dictionary where they keys are the index labels of the pandas.DataFrame to be produces by this function, and values are the columns in dfs missing values should be computed for.

Returns:

pandas.DataFrame

make_table_conditional_flood(floodareasDF, return_periods, waterdepth_labels, extended=False)

Compute the share of addresses that flood for the most likely return period in return_periods, at different maximum inundation depth. Then, for the same subsample of addresses, compute again the shares of addresses that flood at different inundation depths under all less likely scenarios.

Parameters:

• floodareasDF (pandas.DataFrame) – Dataframe from which to compute shares. It is supposed to be the dataframe saved as addressesflooddata.csv in bld.data, restricted so that all observations have value 1 in the column FLOODABLE.

• return_periods (list or tuple) – Can be all or some values in (10, 100, 1000, 10000).

• waterdepth_labels (list or tuple) – Can be all or some values in (“less than 0.5m”, “between 0.5 and 1m”, “between 1 and 1.5m”, “between 1.5 and 2m”, “between 2 and 5m”, “more than 5m”).

• extended (bool) – If True, considers also houses within 500m from flooded areas.

Returns:

pandas.DataFrame

make_table_conditional_flood_all_scenarios(floodareasDF, waterdepth_labels, extended=False)

Compute the share of addresses that flood for the most likely return period in return_periods, at different maximum inundation depth. Then, for the same subsample of addresses, compute again the shares of addresses that flood at different inundation depths under all less likely scenarios. Afterwards, repeat the previous two steps for all the return periods in return_periods, after removing the previous subsample of addresses.

Probabilities are conditional because, in the third step, an address is removed from less likely flood scenarios conditionally on flooding under a more likely one. For example, an address that floods once in 10 years is not included among those that flood once every 100 years. We then follow the (discrete) distribution function of maximum water depth under different scenarios for four separate “cohorts” of addresses (those that flood at most once every 10 years, those that flood at most once every 100 years, those that flood at most once every 1,000 years, those that flood at most once every 10,000 years).

Parameters:

• floodareasDF (pandas.DataFrame) – Dataframe from which to compute shares. It is supposed to be the dataframe saved as addressesflooddata.csv in bld.data, restricted so that all observations have value 1 in the column FLOODABLE.

• waterdepth_labels (list or tuple) – Can be all or some values in (“less than 0.5m”, “between 0.5 and 1m”, “between 1 and 1.5m”, “between 1.5 and 2m”, “between 2 and 5m”, “more than 5m”).

• extended (bool) – If True, considers also houses within 500m from flooded areas.

Returns:

pandas.DataFrame

make_table_exposure(dfs, df_names)

Compute number and share of exposed and not exposed areas for each dataframe in df.

Parameters:

• dfs (list of pandas.DataFrames or geopandas.GeoDataFrames) – Dataframes.

• df_names (list of str) – One for each dataframe in dfs.

Returns:

pandas.DataFrame

make_table_housingtype(df_names, dfs)

Compute number and shares of BAG houses by type, for each dataframe in dfs.

Parameters:

• dfs (list of pandas.Dataframes) – Dataframes.

• df_names (list of str) – Name of dataframes in dfs.

Returns:

pandas.DataFrame

make_table_missings(df_names, dfs, col_dict)

Compute count and share of missing values in dfs for columns specified in col_dict dictionary, as well as t-statistic and p-values for the mean difference of the two missing values distributions.

Parameters:

• df_names (list of str) – Name of two dataframes in dfs.

• dfs (list of pandas.Dataframes) – List of two dataframes.

• col_dict (dict) – Dictionary where they keys are the index labels of the pandas.DataFrame to be produces by this function, and values are the columns in dfs missing values should be computed for.

Returns:

pandas.DataFrame

make_table_target_population(gdf, targetDict)

Compute average values in gdf for variables in targetDict.

Parameters:

• gdf (pandas.DataFrame or geopandas.GeoDataFrame) – Dataframe of interest.

• targetDict (dict) – Dictionary where they keys are the index labels of the pandas.DataFrame to be produces by this function, and values are the columns in dfs average values should be computed for.

Returns:

pandas.DataFrame

make_table_unconditional_flood(floodareasDF, flood_dict, return_periods, waterdepth_labels, extended=False)

Compute the share of addresses that flood, under all scenarios, at different maximum inundation depths.

Probabilities are unconditional because an address is not removed from less likely flood scenario conditionally on flooding under a more likely one. For example, an address that floods once in 10 years is also included among those that flood once every 100 years. As a result, the same address can be included in the share of flooded houses under multiple scenarios.

Parameters:

• floodareasDF (pandas.DataFrame) – Dataframe from which to compute shares. It is supposed to be the dataframe saved as addressesflooddata.csv in bld.data.

• flood_dict (dict) – Empty dictionary to populate. Keys need to represent the flood scenarios, e.g. “1 in 10 years”, “1 in 100 years”, “1 in 1000 years”, “1 in 10000 years”.

• return_periods (list or tuple) – Can be all or some values in (10, 100, 1000, 10000).

• waterdepth_labels (list or tuple) – Can be all or some values in (“less than 0.5m”, “between 0.5 and 1m”, “between 1 and 1.5m”, “between 1.5 and 2m”, “between 2 and 5m”, “more than 5m”).

• extended (bool) – If True, considers also houses within 500m from flooded areas.

Returns:

pandas.DataFrame

melt_friction_vs_clicks(df, topics)

Create long dataframe linking, for each participants, information frictions by topic (in list topics) to whether the participant clicked on the associated topic box.

Script regressions.py

In the folder PYTHON.

Functions shared across the two modules performing checks for the integrity of the experimental randomization, and estimation of the treatment effects.

adjust_pvalues(df, treatments, outcomes, pval_col='P>|z|')

Adjust the p-values in df using the two-stage Benjamini, Krieger, and Yekutieli procedure for controlling the false discovery rate (FDR). Add the adjusted p-values as a new column named “pvalue_fdr_tbsky” to df.

Parameters:

• df (Pandas.DataFrame) – Dataframe of interest. Need to have the column “P>|z|”.

• treatments (list of strings) – Index names to subset. Need to be three.

• outcomes (list of strings) – Column names to subset. Need to be seven.

• pval_col (str) – Name of p-value column. Default is P>|z|.

Returns:

Pandas.DataFrame

create_dictionary_of_covariates(formulas_df, outcomes, add_baseline_beliefs=True, add_het=False, het_covs=[], covs_to_remove=[])

Create dictionary relating each treatment outcome to its set of covariates.

Parameters:

• formulas_df (Pandas.DataFrame) – Dataframe of covariates to use.

• outcomes (list) – List of outcomes.

• add_baseline_beliefs (boolean) – Add outcome-specific baseline beliefs and confidence in beliefs as covariates. Default is True.

• add_het (boolean) – Whether to add variables for heterogeneity analysis to the list of covariates. If True, het_covs need to be a dictionary or a list of variables. Default is False.

• het_covs (list or dictionary) – List of variables for heterogeneity analysis or, if the heterogeneity analysis is outcome-specific, dictionary where keys are outcomes and values are list of variables. Default is empty list.

• covs_to_remove (list or dictionary) – List of variables to be removed from the list of covariates, or dictionary if the variables to remove are outcome-specific. Default is empty list.

Returns:

dictionary.

diff_in_mean(df1, df2, covs)

Compute difference in means along covs between df1 and df2.

load_rlasso_datasets(path_dict)

Load datasets containing outcomes of Rlasso regressions.

logit_treatment_on_response(df, outcome, dep_vars)

Logistic regression where the dependent variable is outcome and the independent variable(s) dep_vars need to contain variable “C(treatment)”.

run_wls_regressions(data, outcomes, depvar)

Run werighted least square regressions of depvar on outcomes, where both are columns of the Pandas.DataFrame data.

ttest(df1, df2, covs)

Compute t-test for difference in means along covs between df1 and df2.

weighted_least_squares_regression(df, outcome, dep_vars)

Run weighted least squares regression on Pandas.DataFrame df, according to formula. df needs to contain columns “WEIGHTS” and “treatment”.

Parameters:

• df (Pandas.DataFrame) – Dataframe of interest.

• outcome (str) – Outcome of formula.

• dep_vars (str) – Dependent variables of linear model.

Returns:

Pandas.DataFrame

wls_on_beliefs_percentile(df, belief, outcome, covariates, percentile, only_keep_interactions=True)

Run weighted least squares with interaction effect over treatment and percentile of prior beliefs.

Parameters:

• df (Pandas.DataFrame) – Dataframe of interest.

• belief (str) – Belief of interest.

• outcome (str) – Outcome of interest.

• covariates (list of str) – Covariates for wls regression.

• percentile (str) – “quartile” or “tercile”.

• only_keep_interactions (bool) – Whether to only store interactions coefficients in the dataframe of results. Default is True.

Returns:

Pandas.DataFrame

wls_with_interaction(df, outcome, covariates, interaction, keep_only_interaction=True, var_to_add=None)

Run weighted least squares with interacted treatment.

Parameters:

• df (Pandas.DataFrame) – Dataframe from which variables are extracted.

• outcome (str) – Name of outcome variable.

• covariates (list of strings) – Names of covariates.

• interaction (str) – Name of interaction variable.

• keep_only_interaction (boolean) – Keep only estimated coefficient of treatment and interacted treatment. Default is True.

Returns:

Pandas.DataFrame

Scripts auxiliary.R and run_rlasso.R

In the folder R.

The script run_rlasso.R uses the R package hdm to select controls for the ATE estimation, via Lasso end Post-Lasso methods for high-dimensional approximately sparse models. The script auxiliary.R contains auxiliary functions to carry out this task.

All the functions in these scripts are duly documented in the .R files. Unfortunately, I cannot export the docstrings with Sphinx (the software that builds this documentation) because the R language is still not supported.