A Nonparametric Variance Decomposition Using Panel Data Fernando Alvarez
Katar´ına Boroviˇckov´a
Robert Shimer
University of Chicago
New York University
University of Chicago
October 20, 2014 Preliminary
Abstract We consider a population of individuals who draw a random variable from an individual-specific distribution that is fixed over time. We propose an unbiased withinbetween variance decomposition using a short panel of two observations for each individual. We illustrate the usefulness of our decomposition with two applications: decomposing heterogeneity versus structural duration dependence in unemployment, nonemployment, and employment durations; and calculating the importance of frictional wage dispersion for labor market outcomes.
1
Introduction
We consider a population of individuals who draw a random variable from an individualspecific distribution that is fixed over time. For example, the duration of an employed worker’s job is a random variable with a distribution that might differ across individuals but is fixed for an individual over his lifetime. The wage of a newly employed worker is a random variable with similar properties.1 We propose that a simple variance decomposition can provide important insight into sources of variance in the whole population. Our basic approach decomposes the variance in aggregate outcomes into two pieces, a within and a between component. The within component is the variance in the outcome for the average individual in the population. The between component is the variance in the mean outcome across individuals. The within component is zero if the outcome is certain for each individual, while the between component is zero if all individuals are identical, or at least have the same mean outcome. In some cases, a zero within component is not a natural benchmark. For example, suppose jobs end with a constant hazard rate. Then there will be some randomness in the duration of a job that occurs for each individual. For such cases, we construct a measure of excess within variance that is positive if the hazard rate is decreasing during an employment spell and negative if the hazard rate is increasing. We also derive unbiased estimators of our variance decompositions using two independent observations of each outcome per individual. Naturally such measures are noisy for each individual, but with a large number of individuals, our measures may be very accurate on average. This is useful because many administrative data sets follow a large number of individuals over a long period of time, yet most individuals experience relatively few employment spells. We illustrate our approach using Austrian social security panel data, but many other administrative data sets contain this type of information. Estimators that rely on individuals who experience many employment spells may offer a selected view of the population of interest. We illustrate the usefulness of our approach through two applications. The first seeks to understand the shape of hazard rates. In Austrian data, we find evidence that heterogeneity is an important driver of the variance in the duration of employment spells, nonemployment spells, and registered unemployment spells. In the last two cases, we also find that the variance is boosted by a decreasing hazard rate, whereby workers who have been employed or not employed for a long time are less likely to exit that state than they were at the 1
In both of these examples, the aggregate state of the economy may influence the outcome, but from our perspective that is part of the randomness an individual faces.
1
beginning of their spell. The evidence for structural duration dependence for the hazard of exiting registered unemployment is more ambiguous. In our second application, we seek to understand why some workers are paid more than others. Again using Austrian data, we find that about sixty percent of the variance in the wage of newly employed workers comes from ex ante heterogeneity, while the remaining forty percent represents some form of luck. This luck is consistent both with search frictions driving wage dispersion and with changes in human capital or the price of human capital pushing wages around over time. The paper proceeds as follows. Section 2 discusses the existing literature that seeks to understand the shape of hazard rates and the extent of frictional wage dispersion. Section 3 demonstrates our variance decomposition both in theory and in practice, when we only observe two independent outcomes per individual. Section 4 analyzes the determinants of the hazard rate of exiting registered unemployment, nonemployment, and employment. Section 5 turns to the determinants of frictional wage dispersion.
2
Related Literature
There is a large literature that seeks to understand the shape of hazard rates. Our paper is most related to those that allow for unobserved heterogeneity. The typical approach to this problem achieves some traction through functional form assumptions. The most common is the proportional hazard assumption (Cox, 1972): the probability that worker i with unobserved characteristic θi and observed characteristic xi finds a job at duration t conditional on not having found one before is θi φ(xi )ht , where ht is the baseline hazard rate, common for all workers. Elbers and Ridder (1982) and Heckman and Singer (1984) show that this model is identified using variation in xi . Honor´e (1993) shows that it is identified by observing multiple spells for each individual worker. Relative to our paper, this research has the advantage of identifying the entire baseline hazard rate, not just whether it has a tendency to be increasing or decreasing. On the other hand, the assumption that the shape of the hazard is the same for all workers is strong and difficult to justify using economic theory (Van den Berg, 2001). Alternatively, a smaller body of literature uses stopping or hitting time models to understand hazard rates. These papers impose that the duration of completed spells has an inverse Gaussian distribution (Alvarez, Boroviˇckov´a, and Shimer, 2014) or a generalization of that distribution (Abbring, 2012). Again these papers rely on particular functional forms to identify the hazard rate. While the functional form has a better theoretical justification—the authors assume that jobs start and end when a latent variable passes a pair of boundaries— 2
the conclusions depend on functional form assumptions. For example, Alvarez, Boroviˇckov´a, and Shimer (2014) assume that the latent variable follows a Brownian motion. Abbring (2012) allows for a more general L´evy process but restricts the extent of unobserved heterogeneity so workers only differ in the distance between the adjustment boundaries. It is doubtful that these functional form assumptions can be substantially relaxed. Again, our approach in this paper is to see how much we can say without making any functional form assumptions. There is also a large body of literature that seeks to understand how search frictions can generate wage dispersion for identical workers. For example, in the partial equilibrium search framework of McCall (1970), a worker samples wages from a fixed offer distribution F and decides when to accept a job. Optimality dictates a reservation wage rule, with the worker accepting any job above the reservation wage. This leads to some observed wage dispersion even if all workers are identical and sample from the age wage distribution. With the introduction of on-the-job search, Burdett and Mortensen (1998) show that a dispersed wage offer distribution F may be consistent with equilibrium even if all firms are identical. A large literature uses the observed cross-sectional wage distribution to estimate versions of this model, typically allowing only for observed worker heterogeneity (Mortensen, 2005) or tightly parameterized unobserved worker heterogeneity (Postel-Vinay and Robin, 2002).2 Using a novel measurement tool, Hornstein, Krusell, and Violante (2011) argue that most of these models cannot generate the extent of wage dispersion observed in the data. Within this literature, our approach is most closely related to the decomposition proposed by Abowd, Kramarz, and Margolis (1999). Those authors use a regression analysis to decompose the variation in log wages into the portion attributable to a worker fixed effect, a firm fixed effect, and a residual. Our approach to the decomposition is less ambitious but also more general. We do not attempt to measure firm fixed effects, because we recognize the possibility that what might be a good job for one worker is a bad job for another worker (Eeckhout and Kircher, 2011). In addition, we measure wages only in new employment relationships, recognizing that these may be different than the wages in existing jobs because workers move up a wage ladder during a long employment spell. In the language of Burdett and Mortensen (1998), we are interested in understanding the wage offer distribution. 2
Postel-Vinay and Robin (2002) assume that workers draw a wage-per-efficiency unit w from a common distribution F but are heterogeneous in terms of their unobserved efficiency θ, earning θw when their wage draw is w.
3
3
Decomposition
We consider an environment with n individuals denoted i = 1, . . . , n. We observe J outcomes yi,j , j = 1, . . . , J, for each individual i. We are interested in environments in which n is large and J is small. To start, we assume each outcome is a random variable drawn independently from a probability distribution function Fi (y). For example, an outcome may be the (log) duration of an unemployment spell or the (log) wage in a job or the (log) time between price changes. The n distribution functions Fi (y) are primitive objects in our environment, but of course might be endogenous in a fully-specified model. Denote the mean and variance of a random variable drawn from the distribution function Fi as µi ≡ σi2
≡
Z
Z
y dFi (y), 2
(y − µi ) dFi (y) =
Z
y 2 dFi (y) − µ2i .
We assume that both of these moments exist. The key assumption behind our interpretation of the data is that Fi (y) is fixed for each individual and common across the J outcomes. At times it will be useful to think that the distribution functions {Fi } are themselves random functions drawn independently from some set of possible distribution functions F with probability measure λ. This allows us to discuss convergence of our estimators as the number of individuals n gets large. Otherwise we do not impose any restrictions on how Fi varies across individuals.
3.1
Theoretical Decomposition
To start, suppose we can observe Fi (y) for all individuals i and all outcomes y. We can measure the population mean and variance: n
1X µ ¯y ≡ n i=1 n
σ ¯y2
1X ≡ n i=1
Z
y dFi (y)
Z
1X (y − µ ¯y ) dFi (y) = n i=1
n
2
4
Z
y 2 dFi (y) − µ ¯ 2y .
We can then decompose the population variance into two components. We define the within component to be the average variance in the population, n
σ ¯w2
n
1X 2 1X = σi = n i=1 n i=1
Z
2
y dFi (y) −
µ2i
.
This represents the variance of the outcome y for the average individual in the population. We define the between component to be the variance of the means in the populations, n
σ ¯b2
n
1X 1X 2 = (µi − µ ¯ y )2 = µ −µ ¯ 2y . n i=1 n i=1 i
This represents the portion of the total variance that comes from heterogeneity in the distribution functions Fi (y) across individuals. Adding these two components gives n
σ ¯w2
+
σ ¯b2
1X = n i=1
Z
y 2 dFi (y) − µ ¯2y = σ ¯y2 ,
so this is an exact variance decomposition.3 To understand this variance decomposition, it is useful to consider two polar cases. In the first case, all individuals draw from the same distribution function, Fi = F¯ for all i. This implies µi = µ ¯ and σi = σ ¯ , so the within variance is equal to the total variance and the between variance is zero when there is no heterogeneity. In the second case, individuals draw from different distributions but each distribution is degenerate, σi = 0 for all i. In this case the within variance is zero, while the between variance is equal to the total variance. This corresponds to the case when the variance in outcomes is entirely attributable to ex ante differences between individuals. Reality is likely to lie between these two extremes.
3.2
Unbiased Estimators
We are interested in a situation in which we cannot observe Fi (y) for any individual. Instead, we observe exactly J = 2 observations yi,1 and yi,2 drawn independently from the distribution Fi (y) for each individual i. We look for unbiased estimators of the within and between variances. To start, we obtain the unbiased estimator of the mean for each individual i and of the 3
These formulae assume that we observe the entire population of n individuals, rather than a sample of size n from a larger population. In the latter case, the unbiased estimates for the population variance and the within and between components are multiplied by n/(n − 1). Since in practice we are interested in cases in which n is large, this distinction proves unimportant.
5
grand mean:
n
1 X yi,1 + yi,2 ˆ¯y = (yi,1 + yi,2). and µ µ ˆi = 2 2n i=1
Similarly, the unbiased estimator of the variance for individual i is J
σ ˆi2 =
1 X (yi,1 − yi,2 )2 (yi,j − µ ˆ i )2 = , J − 1 j=1 2
where we use the formula for a sample variance, dividing by J − 1 rather than J. The unbiased estimator of the total variance is n
ˆ¯y2 = σ
1 X ˆ¯ y )2 + (yi,2 − µ ˆ¯ y )2 (yi,1 − µ 2n − 1 i=1 n
=
1 X 2 2 ˆ¯2y . yi,1 + yi,2 − 2µ 2n − 1 i=1
(1)
We seek to decompose this into the within and between components. To calculate the within component, we simply use that σ ˆi2 is an unbiased estimate of σi2 . Therefore the unbiased estimator of σ ¯w2 is n
ˆ¯w2 = σ
1X 2 σˆ . n i=1 i
(2)
P To calculate the between component, recall σ¯b2 = n1 ni=1 µ2i − µ ¯ 2y and look for the unbiased estimators of µ2i and µ ¯ 2y . Start with µ2i . The variance of µ ˆi conditional on µi is 21 σi2 , or equivalently4 Eˆ µ2i − µ2i = 12 σi2 = 21 Eˆ σi2 , where the expectations operators E recognizes that µ ˆ i and σ ˆi are a random variable conditional on µi and σi (or more precisely the distribution function Fi ). Rearranging terms, µ ˆ2i − 12 σ ˆi2 is an unbiased estimate of µ2i . 4
To prove this, write Eˆ µ2i out explicitly: Eˆ µ2i
=
Z Z
y1 + y2 2
2
1 dFi (y1 )dFi (y2 ) = 2
Z
2
y dFi (y) +
Z
2 ! ydFi (y) ,
where we expand the sum and use the fact that y1 and y2 are independently and identically distributed to simplify each term. The right hand side is equal to µ2i + 21 σi2 , proving the claim.
6
ˆ¯ y conditional on µ Turn next to µ ¯2y . Again, the variance of µ ¯ y is ˆ¯2y − µ Eµ ¯ 2y =
1 2 σ¯ , 2n y
so
1 2 1 ˆ¯y2 . σ ¯y = Eσ 2n 2n
1 ˆ2 ˆ¯2y − 2n σ ¯y is an unbiased estimator of µ ¯ 2y . Finally, take Rearranging terms establishes that µ P the difference between the estimators of n1 ni=1 µ2i and µ ¯2y to get n
σ ¯ˆb2 =
1 2 1 X 2 1 2 ˆ¯ . ˆ¯2y + σ µ ˆi − 2 σ ˆi − µ n i=1 2n y
(3)
ˆ¯y2 with their definitions, it is straightforward to verify that the ˆ¯y , σ ˆi2 , and σ Replacing µ ˆi, µ sum of the within and between estimators is the estimator of the total variance. Thus using just two observations per individual, we can decompose the variance in equation (1) into the within component (equation 2) and the between component (equation 3).
3.3
Consistent Nonnegative Estimators
It is easy to prove that our total and within variance estimators are necessarily nonnegative; however, our estimate of the between variance may be negative. This is true despite the fact that the between estimator is unbiased.5 This issue is not unique to our setup, but our simple formulae highlight the possibility. To see that our estimator of the between variance may be negative, consider an environment in which σi > 0 and µi = µ ¯y for all i. The true between variance is zero, σ ¯b2 = 0, but there is no reason to expect that the between estimator is zero. Indeed, its value will depend on the realized values of the outcomes {yi,j }, which may be arbitrary if the support of the distributions {Fi } are unbounded. For some realizations it will be positive and for some it will be negative. This is not just an issue with our between estimator. Any unbiased between estimator is necessarily negative for some realizations of the outcomes {yi,j }. To prove this, consider an economy in which the distribution of outcomes is common across all individuals, Fi (y) = F (y) for all y, and has full support, F (y) is strictly increasing. The between variance is zero by assumption, so an unbiased estimate that is always nonnegative must have σ ¯ˆb2 = 0 for any realization {yi,j }. Since any outcome {yi,j } can occur with this type distribution, it follows that any unbiased, nonnegative between estimator must always be equal to zero. But now consider another economy in which µi 6= µi′ for some i and i′ . In this case, the between 5
Recall that an estimator is unbiased if the expected value of the estimator is equal to the true value of the parameter being estimated.
7
variance is positive. But since the between estimator must always be equal to zero, it is biased, a contradiction. This leaves us two choices. Either we must accept the possibility that the between estimator is negative or we must use a biased estimator. We advocate the first route, but note that it is easy to construct a nonnegative consistent estimator when we view the individuals as a sample drawn independently from a fixed family of distribution functions F with measure ˆ¯b2 ≥ 0, the within and between estimators are as given in equations (3) and (2). λ. Whenever σ ˆ¯y2 in equation (1) ˆ¯b2 < 0, the consistent within estimator is the total variance σ But when σ and the consistent between estimator is 0. For large n, these estimators converge to the true between and within variances.
3.4
Constant Hazard Benchmark
In some applications, we are interested in decomposing the variance in the duration of completed spells. For example, in labor economics we might examine the duration of unemployment or employment spells. In monetary economics we might look at the amount of time that elapses between changes in the price of a good. In such cases, a constant, common hazard rate is useful benchmark: Fi (t) = 1 − e−ht for some h ≥ 0 and all i and t. This implies that the probability of the spell ending in a short interval [t, t + ε] conditional on it not having ended prior to t is (Fi (t + ε) − Fi (t))ε/(1 − Fi (t)) ≈ hε, a constant hazard rate. Search models frequently assume a constant hazard of both finding and losing a job (Mortensen, 1986), while New Keynesian monetary models frequently assume a constant hazard of a price change (Calvo, 1983). With a constant hazard rate h, the mean duration of a completed spell is µi = µ ¯y = 1/h and its standard deviation is σi = σ¯y = 1/h as well. That is, the coefficient of variation σi /µi is 1. There are two reasons to expect the coefficient of variation to exceed this benchmark in any real world data set. The first is heterogeneity. Not everyone has the same hazard rate, hi 6= hi′ . We capture the role of heterogeneity through the between component of variance. The second we label “excess within variance.” Suppose the hazard rate varies with elapsed duration, hi (t) for individual i at duration t. This corresponds to the cumulative distribution function Fi (y) = 1 − e−
Ry 0
hi (t)dt
.
Then if the hazard rate is nonincreasing and nonconstant, the coefficient of variation exceeds 1, σi > µi (Stoyan and Daley, 1983, pp. 16–19). In other words, for a given mean duration of a spell, a decreasing hazard rate boosts the variance of the spell, creating excess within variance. 8
With this motivation, we can decompose the within variance σ ¯w2 into sum σ ¯e2 + σ ¯c2 , where n
σ ¯e2
1X 2 (σ − µ2i ) = n i=1 i
is the excess within variance, zero if the hazard is constant, positive if it is decreasing, and negative if it is increasing; and n 1X 2 2 σ¯c = µ n i=1 i
is the “constant-hazard within variance,” i.e. the within variance if each individual has a constant hazard rate hi = 1/µi . As before, we obtain unbiased measures of the constant-hazard within and excess within variances using two observations per individual. Recall that µ ˆ 2i − 12 σ ˆi2 is an unbiased estimator of µ2i . Therefore an unbiased estimator of the constant-hazard within variance is n
ˆ¯c2 = σ
1 X 2 1 2 µ ˆ − σ ˆ . n i=1 i 2 i
(4)
The same logic implies that an unbiased estimator of the excess within variance is n
ˆ¯e2 σ
1X = n i=1
3 2 σ ˆ 2 i
−µ ˆ 2i ,
(5)
which of course may be negative even in large samples if hazard rates are increasing. These ˆ¯c2 + σ ˆ¯e2 = σ ˆ¯w2 , and so are again a valid add up to the estimator of the total within variance, σ variance decomposition. We can also decompose the variance of the logarithm of the duration of a completed spell. In the case of a constant hazard, this is always equal to π 2 /6 ≈ 1.64, while it is larger if the hazard rate is decreasing . Thus when we measure the logarithm of the duration of a completed spell, the constant-hazard within variance is σ ¯c2 = π 2 /6 and the excess within variance is
n
σ ¯e2 An unbiased estimator is
1 X 2 π2 = σ − . n i=1 i 6 n
σ ¯ˆe2 =
1 X 2 π2 σ ˆ − . n i=1 i 6
(6)
We use the same notation for the level and logarithmic cases but the context should clarify which is the relevant benchmark. 9
3.5
Draws from Different Distributions
We have assumed so far that the observations yi,1 and yi,2 are independent draws from a common distribution Fi . For some applications this assumption may be reasonable, while in other cases it may not be. For example, the distribution of the duration of a second unemployment spell may be different than the distribution of the duration of the first spell. To capture this, assume that yi,j is a random variable drawn independently from the distribution 2 function Fi,j with mean µi,j and variance σi,j : µi,j = 2 σi,j
=
Z
Z
ydFi,j (y) y 2dFi,j (y) − µ2i,j .
We have so far assumed that Fi,1 = Fi,2 , while now we allow these two distributions to differ. Also define µi and σi2 to be the mean and variance of two draws from the two distribution functions, 1 µi = (µi,1 + µi,2 ) 2 2 Z 1X 2 y 2dFi,j (y) − µ2i . σi = 2 j=1 These are obvious measures of individual means and variances. Note in particular that σi2 exceeds the average variance from the two draws if and only if the means are different: 2
σi2
1 2 1X 2 2 − (σi,1 µ = + σi,2 ) = −µ2i + 2 2 j=1 i,j
µi,1 − µi,2 2
2
The difference reflects the fact that some individual variance comes from the change in the mean between the two spells. We now consider how our estimate of the individual variance, σ ˆi2 = 21 (yi,1 −yi,2 )2 , behaves in this more general environment. We find that it is unbiased if and only if the two distributions have a common mean, µi,1 = µi,2 = µi . To prove this, observe that since yi,1 and yi,2 are independently (but not identically) distributed, the expected value of their product satisfies ZZ 2 Z Y y1 y2 dFi,1 (y1 )dFi,2 (y2 ) = yj dFi,j (yj ). j=1
10
Therefore the expected value of our estimate of the individual-specific variance is 1 2
ZZ
2
1X (y1 − y2 ) dFi,1 (y1 )dFi,2 (y2 ) = 2 j=1 2
=
σi2
+
Z
µ2i
2
y dFi,j (y) −
2 Z Y
ydFi,j (y)
j=1
− µi,1µi,2 =
σi2
+
µi,1 − µi,2 2
2
.
Thus our estimate of σi2 is unbiased if and only if the means of the two draws are equal. Otherwise the estimate is biased upwards. Indeed, the size of this bias is just the difference between the individual variance and the average of that individual’s variances during the two spells: ZZ 1 1 2 2 (y1 − y2 )2 dFi,1(y1 )dFi,2 (y2 ) = 2σi2 − σi,1 + σi,2 . 2 2 Thus we can bound the true variance as: ZZ 1 2 (y1 − y2 )2 dFi,1 (y1 )dFi,2 (y2 ) ≥ σi2 . 2σi ≥ 2
We achieve the lower bound when the two means are the same, while we achieve the upper bound when the variance within each of the spells is zero. The upward bias in our estimates of individual variances will carry over to our estimates of the within variance, which is simply the average of individual within variances. Since it is easy to estimate the total variance using a large sample of individuals, that means that our estimate of the between variance is biased downwards. Unfortunately, while it is possible to reject the null hypothesis that the two draws come from distributions with the same mean, it is impossible to prove the null. In particular, if we observe that the average value of yi,j across individuals i depends on the spell j, we know that at least some individuals have a different mean across the two spells. But even if the average value of yi,j is constant across spells, it may be that it is high for some individuals and low for others. For example, suppose that half the population has a high value of y in their first spell and a low value in their second spell, while for the other half the values are reversed. That is, µi,1 = µi′ ,2 if i ≤ n/2 and i′ > n/2 or i > n/2 and i′ ≤ n/2. Then the mean duration of yi,j is constant across spells, but our estimator of the within variance is biased. Any analysis using multiple spells must implicitly or explicitly address this possibility. It seems reasonable to us that if the mean of yi,1 and yi,2 are the same, then each individual’s outcomes are drawn from two distributions with the same mean. We therefore explicitly assume this in what follows. 11
3.6
Observable Covariates
We now propose an approach to handle cases in which observations yi,j vary systematically with some covariates xi,j . We allow for the possibility that the covariates vary systematically P P across spells, in which case they may explain any difference between ni=1 yi,1 and ni=1 yi,2 .
In particular, if we include the spell j as a covariate, we will always be able to explain any difference in the means. We can also allow for time-invariant covariates. While those can never explain shifts in yi,j , they may explain some of the between variance.
We assume that observation yi,j is the sum of a function φ of the covariates and an idiosyncratic term εi,j , yi,j = φ(xi,j ) + εi,j . In addition, the idiosyncratic term εi,j is a random variable, distributed independently of xi,j and independently across individuals and spells, with some distribution function Fi,j . The mean of Fi,1 and Fi,2 are equal for all individuals i and denoted by µi , while σi2 denotes the variance of a random variable that is drawn with equal probability from Fi,1 and Fi,2 .6 As before, let µ ¯ y denote the population mean of µi and σ ¯y2 denote the population variance: 2
n
1 XX µ ¯y ≡ 2n j=1 i=1
Z
ε dFi,j (ε)
1 XX ≡ 2n j=1 i=1
Z
1 XX (ε − µ ¯y ) dFi,j (ε) = 2n j=1 i=1
2
σ ¯y2
n
2
2
n
Z
ε2 dFi (ε) − µ ¯2y .
Now suppose we know the functions φ and Fi,j . Since yi,j = φ(xi,j ) + εi,j and φ and ε are independent, we can write the variance cross individuals and spells of yi,j as σ ¯y2 = σ ¯x2 + σ ¯ε2 , where σ ¯x2 is the variance of φ(xi,j ), the portion of variance explained by observables, and σ ¯ε is the variance of εi,j , the residual variance which includes unobservable characteristics and idiosyncratic variance. Note that we can add a constant to φ(xi,j ) and subtract the same constant from εi,j for all individuals and spells without changing any of the outcomes or the value of the two variances which makes the mean of these objects uninteresting. We can then divide the idiosyncratic component of variance σ ¯y2 into two components following the same approach as before. Define the within component to be the average 6
Since the mean of random variables drawn from Fi,1 and Fi,2 are equal, the variance of random variables drawn with equal probability from these distributions is equal to the average of the variances of random 2 2 + σi,2 ). variables drawn from each distribution, σi2 = 21 (σi,1
12
variance in the population, 2
n
σ ¯w2
n
1X 2 1 XX = σi = n i=1 2n j=1 i=1
Z
2
ε dFi,j (ε) −
µ2i
.
This represents the variance of the outcome ε for the average individual in the population. Define the between component to be the variance of the means in the populations, n
σ ¯b2 =
n
1X 2 1X (µi − µ ¯ y )2 = µi − µ ¯ 2y . n i=1 n i=1
The within component of the variance can itself be divided up into the standard and excess within variation, with the appropriate definitions dependent on whether we measure duration in levels or logs. To obtain unbiased estimates of the components of the variance decomposition, we propose a two-step approach. First use ordinary or nonlinear least squares to obtain unbiased estimates of the observable component φ(xi,j ) and the idiosyncratic component εi,j . The cross-sectional variance of φ(xi,j ) is the portion of the variance in yi,j attributable to observable heterogeneity. Then proceed as before using the idiosyncratic component εi,j . For each individual, calculate unbiased estimates of the mean, 12 (εi,1 +εi,2 ), and variance, 12 (εi,1 −εi,2 )2 , and use those rather than the yi,j to calculate the between and within estimators as well as the standard and excess within estimators. In particular, suppose the only observable is the spell, xi,j = j. We start with an unbiased estimator of the mean and variance of yi,j , ˆ¯y = µ ˆ¯y2 = σ
Pn
i=1 (yi,1
+ yi,2)
2n ˆ¯y )2 + (yi,2 − µ ˆ¯y )2 ) i=1 ((yi,1 − µ , 2n − 1
Pn
where the second formula corrects for the fact that the mean of y is estimated, reducing the degrees of freedom by 1. We then use OLS to obtain unbiased estimates of the fixed effects. This gives us n 1X φ(j) = yi,j n i=1
for j = 1, 2. Hence the residual is
n
εi,j
1X yi′ ,j . = yi,j − φ(j) = yi,j − n ′ i =1
13
The mean of the residuals is 0 by construction, while an unbiased estimate of the variance of the residuals is Pn (ε2 + ε2i,2 ) ˆ¯ε2 = i=1 i,1 σ . 2(n − 1) Now we reduce the degrees of freedom by 2 since we have estimated two means. Equivalently, ˆ¯y2 as being the mean of the sample variances in the two we can think of the estimate of σ spells, σˆ¯ε2
1 = 2
Pn
ε2i,1 + n−1 i=1
Pn
ε2i,2 n−1 i=1
.
The formulae are obviously equivalent. The variance explained by the spell number is the ˆ¯ε2 . Equivalently, σˆ¯x2 /σ ˆ¯y2 difference between the variance of y and the variance of ε, σ ¯ˆx2 = σ ¯ˆy2 − σ ¯ 2 . As is well-known, this unbiased estimator may be negative.7 is the adjusted R In the second step, we decompose the variance of the residual. We start with the within variance, the average variance of ε2i,j . For each i, an unbiased estimator of σi2 is σ ˆi2 =
n(εi,1 − εi,2)2 , 2(n − 1)
which corrects for the fact that the εi,j are themselves estimates. The unbiased estimator of the within variance is then the average of the unbiased estimators of the individual variances: n
ˆ¯w2 = σ
X 1 (εi,1 − εi,2 )2 . 2(n − 1) i=1
The unbiased estimator of the between variance is simply the difference between the unbiased estimator of the variance in ε and the unbiased estimator of the within variance, ˆ¯y2 − σˆ¯w2 . σ¯ˆb2 = σ Finally, we can also compute unbiased estimators of the constant hazard and excess within variances. We focus on the case in which yi,j measures the log duration of a spell, since if yi,j measures the level of the duration, the linear decomposition may yield nonsensical negative durations. In that case, the constant hazard within variance is always π 2 /6 and the ˆ¯e2 = σ ˆ¯w2 − π 2 /6. estimator of the excess within variance is simply σ 7
To avoid a negative variance decomposition, we could use the unadjusted R2 to estimate the share of variance explained by the spell number. This estimator would be biased but consistent.
14
4
Search and Employment Duration
In our first application, we decompose the variance in the realized duration of job search and employment spells. In the simplest model, all workers find (or lose) a job at a constant hazard rate h. If we measure duration in levels, this implies Fi (y) = 1−e−hy for all durations y ≥ 0 and all individuals i. The mean duration of a spell would be µi = µ ¯y = 1/h and the variance would be σi2 = σ ¯y2 = 1/h2 , giving a tight connection between mean duration and its y
cross-sectional variance. In we instead measure duration in logs, we have Fi (y) = 1 − e−he for all log durations y ∈ R and all i and the variance of y is π 2 /6. The unconditional variance σ ¯y2 is boosted by two additional features of the data. The first is “excess within” variance. For each individual, the hazard of finding or losing a job may decrease with nonemployment duration, boosting the variance relative to the mean (in the levels case) or the unconditional variance (in the logarithmic case). The second is heterogeneity. If mean duration µi varies across individuals, then there is also a nonnegative between component of variance. Our goal is to explore the role of these in determining aggregate variance.
4.1
Data
We measure search and employment duration using data from the Austrian social security registry. The data set covers the universe of private sector workers over the years 1972–2007. It contains information on individual’s employment, registered unemployment, maternity leave, and retirement, with the exact begin and end date of each spell. The use of the Austrian data is compelling for two reasons. First, the data set contains the complete labor market histories of the majority of workers over a 35 year period, which allows us to construct multiple job search spells per individual. Second, the labor market in Austria remains flexible despite institutional regulations, and responds only very mildly to the business cycle. Therefore, we can treat the Austrian labor market as a stationary environment and use the pooled data for our analysis. Almost all private sector jobs are covered by collective agreements between unions and employer associations at the region and industry level. The agreements typically determine the minimum wage and wage increases on the job, and do not directly restrict the hiring or firing decisions of employers. The main firing restriction is the severance payment, with size and eligibility determined by law. A worker becomes eligible for the severance pay after three years of tenure if he does not quit voluntarily. The pay starts at two month salary and increases gradually with tenure. The unemployment insurance system in Austria is similar to the one in the U.S. The 15
Unemployment Exit Hazard
weekly hazard rate
0.08
0.06
0.04
0.02
0
0
10
20
30 40 50 60 70 80 unemployment duration in weeks
90
100
Figure 1: Hazard rate of exiting unemployment during the first two years of registered unemployment, first spell in blue and second spell in red. Results are for all workers with two complete unemployment spells after age 25. duration of the unemployment benefits depends on the previous work history and age. If a worker has been employed for more than a year during two years before the layoff, she is eligible for 20 weeks of the unemployment benefits. The duration of benefits increases to 30, 39, and 52 weeks for older workers with longer work history. Temporary separations and recalls are prevalent in Austria. Around 40 percent of nonemployment spells end with an individual returning to the previous employer. In our analysis here we do not distinguish spells based on whether a worker returns to her previous employer, but our methodology can handle that possibility as well.
4.2
Unemployment Duration
We start by looking at the first two registered unemployment spells for each individual after the age of 25 in our sample. We measure the duration of completed unemployment spells in days. Note that an unemployment spell may end either when a worker finds a job or when she exits the labor market. Our data set contains 1,738,027 workers with mean realized duration 126 days. Figure 1 shows the weekly hazard of exiting unemployment during the two unemployment spells.
16
Unemployment Duration σ ¯ˆy2
σ ¯ˆx2
σ ¯ˆc2
σ ¯ˆe2
σ ¯ˆb2
25+ 25–50
37,004 25,603
— —
23,946 18,547
5,056 1,815
8,003 5,241
25+ 25–50
1.711 1.605
— —
1.645 1.645
-0.187 -0.271
0.253 0.231
25+ 25–50
1.711 1.605
0.000 0.000
1.645 1.645
-0.187 -0.272
0.253 0.232
age levels (no controls)
logs (no controls)
logs (spell controls)
Table 1: Decomposition of the total variance in the level and log of unemployment duration ˆ¯y2 ) into the portion explained by spell number (σ ˆ¯x2 ), the portion consistent with a homoge(σ ˆ¯c2 ), the excess within variance from a decreasing neous worker, constant hazard rate model (σ 2 ˆ¯b2 ). ˆ¯e ), and the between variance from heterogeneity (σ hazard rate (σ We first measure the total variance in duration using equation (1). This is 37,004 days, more than twice the square of the mean. Next we decompose this into the within and between components using equations (2) and (3). These are 29,002 and 8,003 days, respectively. That is, the within component of the variance of duration accounts for 78.4 percent of total variance, while the between component accounts for the remaining 21.6 percent. We find that the “constant-hazard within” variance is 23,946 days, leaving excess within duration at 5,056 days, or 13.6 percent of total duration. Both heterogeneity and a declining hazard rate contribute to the variance in total duration, but the impact of the heterogeneity is larger. The first row in Table 1 summarizes these results. We repeat the same exercise on a more restricted sample of 1,523,149 workers who were at least 25 at the start of the first spell and no more than 50 at the end of the second spell. For these workers, the mean duration of a completed spell is a week and a half shorter at 115 days. Otherwise the second row in Table 1 shows that the decomposition is largely unchanged. In particular, there is some evidence of excess within variance and more evidence of heterogeneity. We next look at the log of duration (third row). The mean of this is 4.17 in our full sample, smaller than the logarithm of mean duration by Jensen’s inequality. The total variance is 1.711, which decomposes into a within component of 1.458 (85.2 percent) and a between component of 0.253 (14.8 percent). Curiously, the within component is smaller than the constant hazard within variance, π 2 /6 ≈ 1.645. Thus the excess within variance
17
is negative and the constant hazard within variance explains the bulk of unemployment duration, 96.1 percent. The results for the more restricted sample of 25–50 year olds in row four are qualitatively unchanged. A robust finding in both samples is that the results in levels point to a decreasing hazard rate while the results in logs point to an increasing hazard rate. This is inconsistent with either a monotonically increasing or a monotonically declining hazard rate. Our numerical simulations suggest that if the hazard rate is hump-shaped, peaking at an intermediate duration, it can generate this empirical pattern. For example, the hazard rate of exiting unemployment may peak near the time when benefits lapse. We can also allow for heterogeneity in means across spells. We find that the first spell last slightly longer than the second. For the full sample, the mean duration is 127.2 days versus 125.3 days in levels, or 4.186 versus 4.162 in logs. These differences are tiny, however, and have little impact on our estimates. For example, in the full sample, we find that the spell number accounts for 0.008 percent of total variance in log duration, with the remaining 99.992 percent accounted for by unobserved worker characteristics and randomness. Not surprisingly, the decomposition of this latter portion is virtually unchanged (Table 1, row 5). The within component is 1.458 (85.2 percent), the between component is 0.253 (14.8 percent), and the constant hazard within variance π 2 /6 still exceeds the within component. The results in the more restricted sample (row 6) are also unchanged. Standard errors of our decomposition are very small. Even though we cannot derive analytical standard errors for our estimates without imposing functional forms on Fi , we can do bootstrapping. We draw 100 random subsamples from the original sample and conduct the variance decomposition for each of them. Each subsamples contains 20 percent of the original sample. We find that the mean within variance share is 78.4 percent with a standard deviation of 0.3 percent for the decomposition in levels. The mean is 85.2 percent and standard deviation is 0.2 percent for the decomposition in logs. And the mean is 85.2 percent with a standard deviation of 0.2 percent for the decomposition in logs with spell controls. Standard errors for the share of excess within variance are similarly small.
4.3
Nonemployment Duration
We also look at nonemployment duration, defined as the elapsed time between two fulltime jobs. Compared to the results for unemployment duration, this has the advantage of clarifying what a worker does at the end of a spell; she starts full-time employment. On the other hand, many nonemployed workers may be uninterested in finding a job, so these results do not speak directly to job search behavior.
18
Nonemployment Exit Hazard
weekly hazard rate
0.12 0.1 0.08 0.06 0.04 0.02 0
0
10
20
30 40 50 60 70 80 nonemployment duration in weeks
90
100
Figure 2: Hazard rate of exiting nonemployment during the first two years, first spell in blue and second spell in red. Results are for all workers with two complete nonemployment spells after age 25. In our analysis here, we again exclude any nonemployment spell which starts before the age of 25. We also exclude spells that include a maternity leave. This gives us a sample of 2,481,277 workers who experience two or more spells. The mean nonemployment duration of 265 days. Figure 2 shows the hazard rate of exiting nonemployment at different durations. We also look at a more restricted sample of 2,291,570 workers whose second spell ends before the age of 50. The mean duration in this sample is 246 days. The results are again broadly similar in the two samples, but quite different than for unemployment duration. First, the variance in spell duration is much greater, nearly ten times greater in levels and twice as large in logs. Second, the excess within variance is always positive and large, accounting for more than half of the total variance in levels and over a fifth in logs. This is consistent with a downward sloping hazard of exiting nonemployment. Third, heterogeneity is also important, particularly when measuring duration in logs. But one thing remains consistent with our earlier findings. Although there is some difference in mean duration between the two spells, 280 days in the first spell and 250 in the second spell in the full sample, this explains virtually none of the overall variance in spell duration.
19
Nonemployment Duration σ ¯ˆy2
σ ¯ˆx2
σ ¯ˆc2
σ ¯ˆe2
σ ¯ˆb2
25+ 25–50
426,063 333,066
— —
107,747 89,805
280,794 213,842
37,522 29,419
25+ 25–50
3.081 3.040
— —
1.645 1.645
0.624 0.604
0.812 0.791
25+ 25–50
3.081 3.040
0.000 0.001
1.645 1.645
0.623 0.603
0.812 0.791
age levels (no controls)
logs (no controls)
logs (spell controls)
Table 2: Decomposition of the total variance in the level and log of nonemployment duration ˆ¯y2 ) into the portion explained by spell number (σ ˆ¯x2 ), the portion consistent with a homoge(σ ˆ¯c2 ), the excess within variance from a decreasing neous worker, constant hazard rate model (σ 2 ˆ¯b2 ). ˆ¯e ), and the between variance from heterogeneity (σ hazard rate (σ
4.4
Employment Duration
We turn next to the duration of employment spells. Many simple job search models assume that jobs end at a constant hazard (Pissarides, 1985; Mortensen and Pissarides, 1994; Shimer, 2005), while in other environments, such as learning models (Jovanovic, 1979) or stopping time models (Alvarez and Shimer, 2011; Alvarez, Boroviˇckov´a, and Shimer, 2014), the hazard of a job ending falls with the duration of the employment spell. Our simple approach can distinguish between these two hypothesis while also allowing for unobserved heterogeneity. We define an employment spell as a sequence of full-time employment without an interruption exceeding 14 days. We impose this cutoff to ensure that if a worker stops one job on a Friday and starts another one on the following Monday or even one week after that, we code this as one long employment spell. As before, our primary sample consists only of workers with two employment spells starting on or after age 25. This gives us a sample of 2,404,142 workers. The mean duration of their spells is 614 days and Figure 3 shows the hazard rate of exiting employment at different durations. The first, third, and fifth rows in Table 3 show the results. The variance of completed spells measured in levels is enormous, nearly 1.5 million days. The majority of this is accounted for by excess within variance, with about ten percent accounted for by heterogeneity. In logs, the story is qualitatively similar, with a large excess within variance but now an even bigger role for heterogeneity. The even-numbered rows in Table 3 show the results for 2,080,675 workers whose second 20
Employment Exit Hazard
weekly hazard rate
0.06
0.04
0.02
0
0
10
20
30 40 50 60 70 80 employment duration in weeks
90
100
Figure 3: Hazard rate of exiting employment during the first two years, first spell in blue and second spell in red. Results are for all workers with two complete employment spells after age 25.
Employment Duration σ ¯ˆy2
σ ¯ˆx2
σ ¯ˆc2
σ ¯ˆe2
σ ¯ˆb2
25+ 1,493,830 25–50 753,584
— —
532,841 294,326
805,009 388,954
155,980 70,304
25+ 25–50
3.501 3.182
— —
1.645 1.645
0.726 0.574
1.131 0.962
25+ 25–50
3.501 3.182
0.001 0.000
1.645 1.645
0.723 0.574
1.132 0.962
age levels (no controls)
logs (no controls)
logs (spell controls)
ˆ¯y2 ) Table 3: Decomposition of the total variance in the level and log of employment duration (σ ˆ¯x2 ), the portion consistent with a homogeneous into the portion explained by spell number (σ ˆ¯c2 ), the excess within variance from a decreasing hazard worker, constant hazard rate model (σ 2 ˆ¯e ), and the between variance from heterogeneity (σ ˆ¯b2 ). rate (σ
21
employment spell also ends by age 50. The mean duration of these employment spells is shorter, 473 days, which is natural since these spells necessarily end more quickly than the average spell. Likewise the variance in the duration of these spells is smaller. But the basic picture is unchanged. Excess within variance and between variance account for much of the overall variance in spell length, indicating that both decreasing hazard rates and heterogeneity are important aspects of employment duration data.
5
Frictional Wage Dispersion
We next turn to another application of our approach, measuring the extent of frictional wage dispersion. We propose measuring frictional wage dispersion as the within component of the variance in wages, while the between component accounts for worker heterogeneity. To the extent that a worker’s characteristics change over time, for example because of human capital accumulation or changes in the price of skills, some of our within component of wages may instead represent time-varying worker characteristics. In this sense, we believe our approach is likely to overstate the extent of frictional wage dispersion. Dating back to McCall (1970), search models of frictional wage dispersion propose that newly-employed workers draw from a fixed wage distribution, taking a job when the wage exceeds their reservation wage. This generates some wage dispersion. Further wage dispersion arises as workers move up the job ladder, as in Burdett and Mortensen (1998). We use our methodology to analyze dispersion of initial wages in a job. We asses what portion of the wage dispersion is attributable to heterogeneity, that is, each worker facing a distribution with person-specific mean, and what portion to the distributions varying across spells.
5.1
Data
We use the same Austrian social security registry data to measure daily earnings. For every worker, the data set contains annual earnings information separately for each employer during the calendar year. Since we also know the start and end date of each job, we can compute average daily earnings. We focus on full-time employees, and call the worker’s average daily earnings her wage. Search-theoretic models of frictional wage dispersion predict that when a worker is newly employed, she draws randomly from the distribution F , while her later wage reflects job-tojob movements up a wage ladder. We therefore average the wage during the first six months of an employment spell following at least 15 days of nonemployment. This is consistent with how we measure employment spells in Section 4.4. We also exclude employment spells that
22
Wage Density
probability density
1.2 1 0.8 0.6 0.4 0.2 0
0
0.2
0.4
0.6
0.8 1 1.2 relative wage
1.4
1.6
1.8
2
Figure 4: Wage density during the first six months of employment, first spell in blue and second spell in red. Wages are expressed relative to annual average wages in the year. Results are for all workers with two complete employment spells after age 25. follow maternity leave, since the worker may simply be returning to her old job. Finally, to control for nominal wage growth, we express this initial wage relative to the average wage in the economy during a given year. We consider workers who have two employment spells that last at least six months and start after age 25. This gives us a full sample of 1,671,348 workers. Figure 4 shows the density of wages in this population. We also restrict the sample to the 1,671,348 workers who start their second spell by age 50. One issue with both of these samples is top-coding, which affects 5.8 percent of spells. This should be borne in mind when reading our results.
5.2
Results
Table 4 decomposes the wage offer distribution in the between (heterogeneity) and within (frictional wage dispersion) components in levels and in logs. In levels, the mean initial wage is 0.845 times the average wage in that year. The first row shows that the variance of initial wages is 0.1077 in our full sample, with 40 percent occurring within workers and 60 percent between. Both channels play an important role in shaping the overall wage distribution, although heterogeneity plays a somewhat larger role.
23
Initial Wage Dispersion σ ¯ˆy2
σ ¯ˆx2
σ ¯ˆw2
σ¯ˆb2
25+ 25–50
0.1077 0.1061
— —
0.0428 0.0427
0.0649 0.0634
25+ 25–50
0.2058 0.2008
— —
0.0861 0.0852
0.1198 0.1156
25+ 25–50
0.2058 0.2006
0.0001 0.0002
0.0859 0.0848
0.1199 0.1158
age levels (no controls)
logs (no controls)
logs (spell controls)
ˆ¯y2 ) into the Table 4: Decomposition of the total variance in the level and log of initial wages (σ ˆ¯x2 ), the within variance (σ ˆ¯w2 ), and the between variance portion explained by spell number (σ ˆ¯b2 ). (σ We then look at log wage dispersion. Although the variance in log wages is bigger, the split is roughly 40 percent within, 60 percent between. We also find little difference in the log wage between the two spells, with the log wage in the second spell 0.002 higher than in the first spell. Therefore accounting for spell number does not affect our results. The results in our restricted sample of workers whose second employment spell starts before the age of 50 are the same.
References Abbring, Jaap H., 2012. “Mixed Hitting-Time Models.” Econometrica. 80 (2): 783–819. Abowd, John M., Francis Kramarz, and David N. Margolis, 1999. “High Wage Workers and High Wage Firms.” Econometrica. 67 (2): 251–333. Alvarez, Fernando, Katar´ına Boroviˇckov´a, and Robert Shimer, 2014. “Decomposing Duration Dependence in the Job Finding Rate in a Stopping Time Model.” University of Chicago Mimeo. Alvarez, Fernando, and Robert Shimer, 2011. “Search and Rest Unemployment.” Econometrica. 79 (1): 75–122. Burdett, Kenneth, and Dale T. Mortensen, 1998. “Wage Differentials, Employer Size, and Unemployment.” International Economic Review. 39 (2): 257–273. 24
Calvo, Guillermo A., 1983. “Staggered Prices in a Utility-Maximizing Framework.” Journal of Monetary Economics. 12 (3): 383–398. Cox, David R., 1972. “Regression Models and Life-Tables.” Journal of the Royal Statistical Society. Series B (Methodological). 34 (2): 187–220. Eeckhout, Jan, and Philipp Kircher, 2011. “Identifying Sorting: In Theory.” Review of Economic Studies. 78 (3): 872–906. Elbers, Chris, and Geert Ridder, 1982. “True and Spurious Duration Dependence: The Identifiability of the Proportional Hazard Model.” The Review of Economic Studies. 49 (3): 403–409. Heckman, James, and Burton Singer, 1984. “The Identifiability of the Proportional Hazard Model.” Review of Economic Studies. 51 (2): 231–241. Honor´e, Bo E., 1993. “Identification Results for Duration Models with Multiple Spells.” Review of Economic Studies. 60 (1): 241–246. Hornstein, Andreas, Per Krusell, and Giovanni L. Violante, 2011. “Frictional Wage Dispersion in Search Models: A Quantitative Assessment.” American Economic Review. 101 (7): 2873–98. Jovanovic, Boyan, 1979. “Job Matching and the Theory of Turnover.” Journal of Political Economy. 87 (5): 972–990. McCall, John Joseph, 1970. “Economics of Information and Job Search.” The Quarterly Journal of Economics. 84 (1): 113–126. Mortensen, Dale T., 1986. “Job Search and Labor Market Analysis.” in Orley Ashenfelter, and Richard Layard (ed.), Handbook of Labor Econoics, vol. 2, chap. 15, pp. 849–919. Elsevier. Mortensen, Dale T., 2005. Wage Dispersion: Why are Similar Workers Paid Differently?, MIT press. Mortensen, Dale T., and Christopher A. Pissarides, 1994. “Job Creation and Job Destruction in the Theory of Unemployment.” Review of Economic Studies. 61 (3): 397–415. Pissarides, Christopher A., 1985. “Short-Run Equilibrium Dynamics of Unemployment, Vacancies, and Real Wages.” American Economic Review. 75 (4): 676–690.
25
Postel-Vinay, Fabien, and Jean-Marc Robin, 2002. “Equilibrium Wage Dispersion with Worker and Employer Heterogeneity.” Econometrica. 70 (6): 2295–2350. Shimer, Robert, 2005. “The Cyclical Behavior of Equilibrium Unemployment and Vacancies.” American Economic Review. 95 (1): 25–49. Stoyan, Dietrich, and Daryl J. Daley, 1983. Comparison Methods for Queues and Other Stochastic Models, John Wiley & Sons. Van den Berg, Gerard J., 2001. “Duration Models: Specification, Identification and Multiple Durations.” in James J. Heckman, and Edward E. Leamer (ed.), Handbook of Econometrics, vol. 5, chap. 55, pp. 3381–3460. Elsevier.
26
