The Relationship between Academic Research and Teaching Quality∗ Mauro Sylos Labini†
Natalia Zinovyeva‡
December 22, 2011
Abstract This paper analyzes the relationship between research and teaching quality in the Italian system of higher education. We find that a few self-reported measures of students’ teaching satisfaction are positively correlated with department-level indicators of academic research quality based on expert evaluation scores and bibliographic information. To check whether this correlation stems from unobserved university characteristics, we exploit within-university across-departments variation in research quality and find that the association with students’ satisfaction is still positive. Finally, we rely on department-level variation of research productivity across time and find that it is positively correlated with cross-cohort variation in graduates’ labour market outcomes. If the latter is taken as a proxy for teaching quality, this evidence suggest that enhancing the quality of academic research helps as well to increase the quality of teaching.
JEL Classification: I23 Keywords: Academic research, teaching quality, labour market outcomes
∗
We would like to thank participants of the workshop ”Labor Market for Scientists and Engineers” in Maastricht, ESSID workshop on Economics of Science in Dubrovnik, EALE conference in Oslo, seminars at the University of Aarhus, Bocconi University, University of Pisa, University of Strasbourg, Carlos III University and FEDEA for their comments on an earlier draft of this paper. The empirical analysis of this paper would not have been possible without the data and the help provided by members of the KEINS project, and particularly Francesco Lissoni and Bulat Sanditov, the Conference of Italian University Rectors (CRUI), and particularly Elena Breno, and ISTAT (the Italian Statistical Office). The econometric analysis was carried out at the ADELE Laboratory. † IMT Lucca Institute for Advanced Studies, email: ... ‡ FEDEA, email:
[email protected]
1
1
Introduction
In continental Europe a relevant portion of top quality research is performed by non-university institutions. For example, French CNRS (National Institute for Scientific Research), INRA (National Institute for Agricultural Research), and the Institute Pasteur are among the five French institutions that employ the highest number of highly cited researches (see Table 1). Similarly, a high number of the best German scientists is affiliated with the Max Plank Institutes. On the contrary, in the US and the UK research universities are the top institutions in terms of research performance. Given the superiority of Anglo-Saxon universities in most international rankings,1 it has been debated whether continental Europe should promote a shift of its top scientists from only-research institutions to research universities (Mowery and Rosenberg, 1993; Mas-Colell, 2003; Dosi et al., 2006) and whether university funding should be stronger related to academic research output. Assessing whether research excellence and teaching quality are positively correlated is important for the above question and motivates the present paper. The common wisdom is that teaching (especially graduate) and research activities are strongly intertwined. For instance, Mas-Colell (2003) argues that “from the standpoint of the social research objective it would be a considerable waste not to demand research from a collective, the university teaching staff, singularly prepared for the task [and f]rom the standpoint of the training objective [...] the potential for excellent training can be much reinforced by an environment of creativity where the frontiers of knowledge are being relentlessly pressed forward” (p.13). Similarly, Mowery and Rosenberg (1989) note that many research universities “exploit a great complementarity between research and teaching. Under the appropriate set of circumstances, each may be performed better when they are done together” (p.154). Note that the causality can go both ways. In fact, on the one hand, being on the research frontier is important to effectively teach up-to-date courses and novel methodologies. On the other hand, designing advanced courses may help in tackling the relevant questions and in 1
See for example the so-called Shanghai Ranking at http://www.arwu.org.
2
envisaging a broader picture of a given research area (Becker and Kennedy, 2005). There exist, however, several objections to this view. Most importantly, teaching and research could well be substitutes if one considers faculties’ time-, effort-, and funds-allocation constraints. Moreover, they may also require different underlying abilities and skills. The overall balance between the above forces is ultimately an empirical question. Although there exists an extensive empirical literature addressing the issue, most investigations seem to suffer two major problems: first, most of them exploit across-departments variation within a given university and their results are thus difficult to generalize. Second, since most of studies rely on cross-sectional datasets, they cannot exclude the possibility that correlation between teaching and research quality is affected by the omitted variables bias. This paper relies on a unique data, which contains teaching and research quality indicators for of the whole Italian university system. We first exploit across-departments variation in research quality to check whether it is correlated with a few self-reported measures of students’ satisfaction with their professors. We find a positive and statistically significant association, even controlling for a number of individual and university characteristics. To check whether this correlation stems from unobserved university characteristics, we add university dummies and thus restrict to within-university across-departments variation in research quality. We find that the correlation with students’ satisfaction is still positive. We then use graduates’ employment outcomes as an indicator of teaching quality and exploit variation of research productivity across time. Our results show that variation in research productivity across time is positively and significantly associated with students occupational outcomes. The rest of the paper is organized as follows. In Section 2 we summarize the background literature addressing the issue of teaching-research relationship. Section 3 describes the data used for empirical analysis and section 4 explains the main steps of our empirical strategy. The results of our empirical analysis are discussed in section 5. Section 6 concludes.
3
2
The pro’s and con’s of research-teaching complementarity and the existent empirical evidence
There are good reasons to believe that teaching and research are complements. Given that university teaching staff is singularly prepared for research, it seem natural to demand some research activity from them. However, from the standpoint of the training objective it is unclear whether a goal of pushing forward knowledge frontiers could reinforce the potential for excellent teaching. The conflicting arguments may be summarized as follows. On the positive side, research is likely to help individual professors in mastering both the state of the art and the advances of their discipline. Consequently, those instructors involved in research activities are better suited to update and ameliorate the content of their courses. The benefits for the quality of their classes and therefore for students are self-evident. On the negative side, faculties may face a basic trade-off in the effort, time and budgets they devote to teaching compared to the one they devote to research (Marsh and Hattie, 2002). The strength of these trade-offs depends largely on the incentive schemes adopted by universities. Often universities (and especially public universities) adopt incentive schemes that are mostly based on research output2 and, as a consequence, researchers might neglect their teaching duties especially when they are young and are seeking tenured positions. Still, in a different environment where the important source of academic research funding is associated to the teaching outcome provided by professors, the trade-offs might become less severe. In fact, in this environment, in order to improve their research output, professors would need to improve their teaching quality as well. Gautier and Wauthy (2007) argue that this mechanism might be particularly effective in multidepartment universities. The authors suggest that strategic complementarity between teaching and research for university as a whole combined with the incentive scheme generating between-department yardstick competition for research funds may promote additional efforts in both dimensions. 2
See Mas-Colell (2003) for an interesting discussion on why this is likely to be the case.
4
Hattie and Marsh (1996) perform a meta-analysis summarizing existent evidence (mainly for the US) concerning the teaching-research relationship. They review 58 studies published between 1949 and 1992 and find that on average these works pinpoint to the existence of a small positive correlation between different indicators of teaching and research. However, there is a substantial variance across the studies. One of the reasons for big variance in the results is heterogeneity of environments in which the relationship between research and teaching was observed. Most studies rely on cross-sectional datasets often specific to a single university. The level of analysis (individual vs. department level) can be another factor driving variation in results. In fact, the apparent complementarity between teaching and research at the department level paradoxically can emerge when the two activities are actually substitutes at the individual level. This would occur if some departments adopt a specialized internal organizational form, in which some academics are mainly involved in research and others perform the major part of teaching. Another important issue is the measurement of teaching and research quality, which differs across studies. Hattie and Marsh (1996) find that 93% of studies use bibliometric indicators to evaluate research (the number of publications, the number of received citations, or other estimates of productivity such as weighted sums of chapters, books, articles, etc.), whereas other studies rely on information on the amount of obtained research grants. As far as teaching quality measurement is concerned, most studies rely on students’ evaluations (80%). Note that this method has some drawbacks: in many cases students lack the basic capabilities to evaluate their instructors. Students might prefer easygoing professors or applied courses, irrespectively of the quality of the didactics. They might simply have limited information about the teaching quality of professors in other institutions, at least, before they graduate and get to the labour market. Therefore they might tend to provide the evaluations relative to other professors in their department. In this case the comparison of student evaluations across departments would fail to reflect the department differences in teaching quality. An
5
alternative indicator is peer evaluation by colleagues, which is used by 19% of the investigations reviewed by Hattie and Marsh. Still, this indicator might be also subject to reputation biases if an eminent researcher is more likely to be considered a better teacher. The performance of graduates in the labour market and the degree of their job satisfaction might be another important and more objective measure of university teaching quality. There exists a broad literature analyzing the effect of college characteristics on graduates’ labour market performance. Most of empirical studies analyze the effect of expenditures per student and selectivity of colleges (Black and Smith, 2004; Brewer, Eide, and Ehrenberg, 1999; Dale and Krueger, 2002). At the same time, the evidence on the relationship between the quality of university research and students’ careers after their graduation is scarce. None of the studies included in the meta-analysis of Hattie and Marsh (1996) studied the relationship between academic research and graduates’ labour market performance. The article of Urwin and Di Pietro (2005) is among few existent studies addressing the relationship between academic research and graduates’ careers. The authors show that in the UK the research quality of institutions as measured by the scores issued by the Research Assessment Exercise (RAE) is positively related to the labour performance of graduates from graduate schools (i.e. Master and PhD programs), conditional on a range of individual and department-level controls. This result might seem intuitive given that at the postgraduate level many students target research-oriented careers and thus are likely to benefit from the research experience of their teachers. At the same time, the evidence at the undergraduate level provides rather conflicting results. For Northern Ireland, McGuinness (2003) finds only insignificant positive relationship between the RAE scores of institutions and graduates labour market performance. In the case of the US universities, James, Alsalam, Conaty, and To (1989) observe that the presence of a PhD program at the department (which proxies the intensity of the department research activity) has no positive effect on the labour market outcomes of graduates in Bachelor degrees. For Italy, Di Pietro and Cutillo (2006) find that better position of the institution in the ranking released by one of the leading Italian
6
newspapers La Repubblica is associated with higher earnings and lower overeducation of graduates. The scarcity of evidence on the effect of university research on graduates labour market success mainly reflects a lack of available datasets appropriate to address this issue. Due to the same reason the consistency of above findings is typically conditioned on several strong assumptions. First, it is assumed that individuals of the same age, sex, achieved highschool grade and few other observed characteristics, choose randomly where to study. The potential existence of differences in the unobservable characteristics of students that attend different colleges could however be originated either by the existence of self-selection of better and more motivated students into better colleges, or by, paradoxically, low self-selection combined with high unobservable geographical heterogeneity in individual characteristics. Second, the existence of unobserved heterogeneity across universities is typically ruled out. Still, departments’ research quality is very likely to be related to university ‘prestige’ that makes university more attractive for both good students and good professors. In this study we analyze the relationship between research and teaching quality of university departments relying on a unique dataset, which allows us to overcome many of the problems discussed above. First of all, our dataset covers information for the whole Italian university system in the period between 199? and 2001. This gives high internal validity to our results. Moreover, Italian higher education shares many features with other countries in continental Europe, which makes the evidence presented in this study relevant beyond Italian university system. One of these features concerns the incentive system faced by universities and professors. During analyzed period Italy had quite homogenous incentive system for professors and institutions. Distribution of funds across public universities was mainly done on the historical basis and was not linked to any indicator of teaching quality. Presumably, promotion decisions were mainly done based on candidates’ research production, even though it is widely accepted that in practice networking was at least as important for moving along the career ladder. Another feature of this system was an obligation of all professors of a
7
given rank to provide a certain number of hours of teaching as determined by the Ministry of Education. This latter feature excludes the possibility of specialization of professors across teaching and research activities. Second, we observe several measures of departments’ research and teaching quality. We proxy the quality of academic research by the number of publications, the number of patent applications done by academic inventors, and the score received from the Italian Ministry expert evaluation committee (analogous to British RAE). To capture departments’ teaching output, we rely on the data from the three waves of Italian triennial national survey of graduates. We proxy teaching quality both by teaching evaluations and labour market performance of graduates. The dataset also includes extensive list of individual background characteristics (family background, high-school grade, high school type, province of origin, etc.) and institutional controls (department size, student-per-professor ratio, average professor age, university ownership). Finally, differently from previous studies, we rely on over-time variation in departments’ research quality and cross-cohort differences in graduates’ success rate in the labour market. Variation in the research quality of professoriate comes from the incorporation of new staff due to the expansion of Italian university system during this period. Exploiting over-time variation allows us to rule out the effect of all unobserved university and cohort characteristics that were time-invariant during analyzed period. Most importantly, this allows us to exclude the possibility that the correlation between departments’ research and teaching outcomes is driven by university reputation and prestige, as soon as we are ready to accept that university prestige is rather inert relatively to changes in professors’ research production.
3
The data
The empirical analysis of this paper is based on a very detailed dataset concerning Italian university graduates which allows to observe their socioeconomic background, high school
8
grades, university performance, satisfaction with teaching quality and, finally, their outcomes in the labour market and in professional qualification exams. This dataset is combined with the official statistical data on academic institutions where individuals studied, expert evaluations of academic research quality as well as various bibliographic measures of the research output in these institutions. More specifically, data about graduates’ labour market performance comes from three distinct but almost identical surveys named Indagine Inserimento Professionale Laureati (Survey on University-to-Work Transition) run in 1998, 2001, and 2004 on individuals that graduated in 1995, 1998, and 2001 respectively. The target samples consist of 25716 individuals in 1998, 36373 individuals in 2001, and 38470 individuals in 2004. They represent respectively the 25%, 28.1%, and 24.7% of the total population of university graduates in Italian universities. The response rates were of 64.7%, 53.3%, and 67.6% for a total of 17326, 20844 and 26006 respondents. In all three years the sample is stratified according to sex, university and obtained degree and in the analysis below all estimations are performed using stratification weights. We exclude from the sample graduates from physical education studies and from the so-called “laurea primo livello”, since they were surveyed only in the 2001 edition (501 and 475 observations respectively). The surveys provide information on (i) individual characteristics that are predetermined to college choices and outcomes, (ii) college-related individual indicators and (iii) labour market outcomes. The first set of variables includes information on the individual sociodemographic background such as gender, nationality, number of siblings, province of residence before college enrollment, parents’ education and employment when respondent was around 14 years old, the situation of military service obligations before attending university and self-reported high school curricula – final high school grade and type of school attended. The second includes university-related indicators: the type of degree and university attended, educational outcomes – i.e. final grade obtained and the number of years spent for the com-
9
pletion of the degree3 – and additional information such as occupational status during studies, switching the degree followed, attainment of an other degree. Information on students’ satisfaction with teaching quality is available for the 1998 survey. Third, the survey collects self-reported information about a number of occupational outcomes three years after graduation. Among others, it is possible to observe whether the graduate is employed, whether university degree was a pre-requisite for obtaining the job, wage and several indexes of job satisfaction. Table 2 shows the descriptive statistics for the key individual variables.4 We have also obtained official data on several college characteristics. Note that in Italy the teaching activity is concentrated around faculties (facolt`a), whereas research activity is coordinated by departments or, translating literally from Italian, by scientific sectors (settori scientifico disciplinari), which overlap, but do not necessarily fully coincide. Based on the title of graduates’ degrees, we assigned each individual to a corresponding scientific sector (see Appendix for used association between degrees and scientific sectors). In this way we were able to match individuals with information on both teaching and research inputs and outputs available in the university in a specific field of study. Specifically, we have collected information from different issues of ISTAT bulletins La universit`a in cifre and Lo Stato dell’Universit`a on the total number of students enrolled in the faculty, the number of first-year students and the total number of students that graduated each year. The official reports of Italian Ministry of Education, University and Research (Ministero dell’Istruzione, dell’Universit`a e della Ricerca – MIUR) also provide information at the level of scientific sector on the number and age of professors. The descriptive statistics for these variables are presented in Table 3. 3
In Italy the final grade is calculated as the sum of the grades obtained by the graduate during her courses plus the grade received for the so-called degree dissertation (tesi di laurea). Any student whose final grade is higher than 110 obtains what is known as “110 e lode”. Also note that in the Italian education system in the analyzed period students were not constrained either in time or in the number of trials for passing exams. 4 The unemployment rate for graduates in our sample is 14.7 percent. It is consistent with the OECD 2003 data suggesting that 13.6 percent of Italian graduates aged 25-29 not being in education are unemployed. Italian graduates experience disadvantage in terms of early performance in the labour market relatively to the rest of individuals of their age: the overall unemployment rate among individuals aged 25-29 is 10.4 percent (OECD, Education at a Glance 2005).
10
Finally, the above dataset was combined with three different measures of research activity at the level of university scientific sectors. The most reliable measure of academic research quality in our analysis comes from the expert evaluation of academic research output during 2001-2003 performed by the expert committee under the support of MIUR ( Valutazione triennale della ricerca - VTR 2001-2003 ). This evaluation was a first national experiment of this kind analyzing research output of 77 universities in 20 scientific areas. The evaluation procedures consisted of the following stages. First, each researcher at university was asked to present her two best “scientific outputs” produced during the period 2001-2003. The definition of a “scientific output” includes such research outputs as books, book chapters, articles, patents and other results of applied research, projects, designs, exhibitions and others (in total 17,329 “scientific outputs” were considered). Second, there were elected 6,661 international experts, who had to evaluate the quality of research outcomes in 151 panel-lists defined for different subfields. Each “scientific output” was evaluated by at least 2 experts according to the following criteria: quality, relevance, originality/innovation and internationalization and/or international competitive potential. Finally, each area panel has produced a report and a ranking list. The second source of information on university research outcomes comes from the Thompson ISI dataset on publication and received citations cleaned and aggregated by the Conference of Italian University Rectors (Conferenza dei Rettori delle Universit`a Italiane - CRUI ) to the level of university scientific disciplines (see Breno et al. (2002) for the description of the data cleaning and aggregation). This information is updated on July 2000 and concerns the period of 1995-1999. In the data provided, CRUI has narrowed the ISI classification of disciplines to fit 14 broad disciplinary sectors, which are used by the Italian Ministry of Education, University and Research (Ministero dell’Istruzione, dell’Universit`a e della Ricerca – MIUR) for financial and policy decisions. However, the data published by CRUI gives information concerning only 9 disciplinary sectors that exclude humanities and social sciences.5 In 5
The 9 sectors considered by CRUI are Mathematics and Informatics, Physics, Earth Sciences, Biosciences, Pharmacy and Chemistry, Medicine and Surgery, Veterinary and Agriculture, Construction Engineering and
11
order to be able to analyze the relationship between academic research and the performance of graduates from all three available cohorts, we also obtained the data on ISI publications in 2001 and their citations received until March 2009, following the procedure described in Breno et al. (2002). Finally, our third measure of academic research output is the number of patents registered within a three-year period in the European Patent Office by inventors from a given university department, as well as the number of citations received consequently by these patents as observed in 2004.6 The academic research output is measured with the use of the KEINS dataset on academic patenting (Lissoni, Sanditov and Tarasconi, 2006). In contrast to the US case, in Italy during analyzed period universities did not generally retain the property rights on inventions done by their researchers. Often “IPRs over inventions derived from sponsored research programmes were left to the sponsors (see Balconi, Breschi, and Lissoni, 2004). This has reduced substantially the precision, with which university patenting activity could be measured. However, the KEINS EP-INV database matches the names of the inventors of the patents with a list of university professors. Thanks to this methodology, the KEINS database includes not only any patent owned by universities, but also all patents that involve university scientists, whether the patents are owned by firms, public research organizations, universities, or the scientists themselves.7 The descriptives for academic research indicators are in Table 4. The timing of the available information on graduates’ labour market outcomes, evaluation of teaching quality and university research indicators is described in Table 6. The propensity to publish, to patent and to cite other authors’ work differs substantially across the disciplines. In our following analysis we standardize our indicators of academic research to have zero mean and unit standard deviation within each scientific discipline and year. Table 5 shows the correlations between the standardized measures: scores assigned Architecture, and Industrial Engineering. 6 More precisely, the database includes all patent applications that passed a preliminary examination in the EPO. The assigned date of the patent is the priority date, which is the date of the first filing world-wide. 7 The list of professors, which KEINS database match the list of university inventors, includes all university professors in 2000. This means that the precision with which academic patents are identified declines the further we go back in time in patent series.
12
to university departments by expert evaluation committees, the number of publications (or publication citations) per professor and the number of patents (or patent citations) per professor. All correlation coefficients appear to be positive and significant at 1 percent level. The highest correlation is observed between the number of citations received by an average professor in the department and VTR scores (32 %) and the lowest correlation is between the number of citations received by academic patents and VTR scores (11%).
4
Empirical Analysis
We analyze the effects of academic research in two steps. First, we consider the crosssection of 1995 graduates and investigate whether faculties’ research activity is correlated with students’ evaluations of professors’ competence in the subject of their teaching. The results of this step should indicate whether there is a potential complementarity relationship between research and teaching quality perceived by students. On the second step we use the information on repeated cross-sections of 1995, 1998 and 2001 graduates in order to analyze the effect of academic research on graduates’ labour market performance.
4.1
Graduates’ satisfaction with teaching
To capture the relationship between the quality of research carried out by professors and students’ evaluation of professors’ competence we estimate the following model:
T = α + Xβ + Rfu γ + Ufu δ + Dd η +
(1)
where T represents satisfaction with teaching quality reported by an individual, who graduated in a field of study (or discipline) f at university u, X is a set of individual characteristics predetermined with respect to the choice of university and department. We consider students’ gender, age group, province of origin, parents’ occupation and education, the high school final grade and the type of high school attended. Rfu is a vector of academic research indicators 13
for discipline f at university u. Dd - contains the dummies for each degree d, where degree is a more precise indicator of a graduate’s specialization than discipline f .8 We cluster standard errors by university and discipline. The first look on the unconditional relationship between expert evaluation scores and average students’ satisfaction with professor competence. As Figure 1 suggests, there is an apparent positive relationship between these two variables almost in all scientific disciplines. (see Table 7 column 1)
9
As column 2 suggests, most of this relationship is due to the
regional differences: the inclusion of the dummies for Italian macro-regions decreases the estimate twofolds. In the present paper we estimate a fully parametric model; nevertheless, we allow a large number of regressors – including the ones capturing individual characteristics – to enter in a non-linear way, mainly through the creation of a series of dummy variables. We perform a reduced-form estimation which is widely used in the literature:
T = α + Xβ + Rfu γ + Ufu δ + Dd η +
(2)
where T represents satisfaction with teaching quality expressed by an individual, who graduated in field of study (or discipline) f at university u, X is a set of individual characteristics predetermined with respect to the choice of university and department. We considered students’ gender, age group, parental background in terms of occupation and education, the mark reported in the high school graduation exam and the type of high school attended. We also controlled for students’ province of origin. Rfu is a vector of academic research indicators defined for discipline f at university u. Dd - contains the dummies for an exact degree d obtained by an individual, where degree is a more precise indicator of a graduate’s 8
In most cases graduates of the same degree and the same university attend almost all courses together. Moreover, the degree content varies little across universities. 9 In order to facilitate the interpretation and the exposition of the results, this paper presents OLS estimates for all analysis that follows. Given the discrete nature of student satisfaction measures, we also estimated the corresponding models using ordered probit. The results are qualitatively similar and available upon request.
14
specialization than a filed of study f .10 Ufu refers to other observable characteristics of a university department where a graduate received her diploma. This characteristics include both time-invariant characteristics – location and ownership – and time-variant characteristics – student-per-professor ratio, professors’ age and the (log)number of professors. Controlling for department size as well as for average class-size is important for the following reason. In Italy (similarly to many other European countries) public education is almost free of charge and the admission to universities is open almost for all degree programs. No selectivity constraints imposed on enrollment make the actual enrollment size be heavily demand driven. However, there exist rigidities and delays in the universities’ capability to adjust its inputs to variations in enrollment. Thus a sudden increase in the number of students can affect the students-per-professor ratio and consequently the time which professors have available for research. Given that the information concerning student satisfaction with the quality of teaching is available only for the 1995 cohort of graduates, equation (2) is a cross-section estimation model. This means that our results might be still confounded by some unobserved characteristics at the university or department level that are related both to the quality of teaching and research. We perform a robustness check including university dummies into equation (2). Inclusion of these controls reduces substantially the exploited variability of academic research. The positive correlation between the quality of academic research and the quality of teaching can have several interpretations. First of all, better researchers may be performing better as teachers, because they are able to provide more competent and up-to-date knowledge to their students. However, the positive correlation between teaching and research at the department level does not necessarily imply an a priori professor-level complementarity between the two activities. Professor performance on both dimensions might depend on the incentive schemes 10
In most cases graduates of the same degree and the same university attend almost all courses together. Moreover, the degree content varies little across universities.
15
present at the department or university level. However, in Italy the quality of teaching does not affect directly neither professor salary nor career development. Alternatively, in the extreme case when individually teaching and research activities are substitutes and at the same time universities have differential possibilities to exploit specialization, the departments in which professors have an opportunity to specialize, overall would be able to provide both better research and teaching outcomes. However, in Italy the teaching loads of professors are set by the state law and cannot be modified internally. Still, departments differ in the proportion of faculty on non-tenured positions. Professors with fixed contracts are often hired by departments for specific uncovered teaching slots. In principle, such departments might benefit from their relative specialization with respect to other departments in case there exist individual-level substitutability between teaching and research. We collect information about the proportion of non-tenured professors in each department as a proxy of a rather specialized teaching activity and test whether departments that have more professors with fixed contracts are also better in teaching and research quality. Results presented below are unaffected by the inclusion of this control. Finally, we estimate the effect of academic research quality on graduates’ labour market performance. The data on graduates’ labour market performance is available for all cohorts. This allows us to capture the over-time variation in the effect of academic research and other department characteristics on graduates’ performance. First, we estimate the determinants of graduates’ labour market performance L using a pooled cross-section estimation:
L = α + Xβ + Rtfu γ + Utfu δ + Dd η + Dt ζ +
(3)
where Dt – is a set of time dummies for each cohort of graduates. Model (3) will provide inconsistent estimates if research quality and labour market outcomes are correlated with other unobserved university characteristics. For instance, location characteristics might affect both the fact that good researchers choose to work in particular universities and the fact that graduates of these universities are more successful in the 16
labour market. The problem of self-selection might also be an issue. Prestigious universities are likely to attract better students who after graduating will also tend to earn more than graduates or similar quality from less prestigious universities. Prestigious universities can also offer better teaching and research conditions for academics themselves, and this might create another source of self-selection bias. If the observed individual and department characteristics X fail to account for all dimensions of student quality and department prestige, the problem of self-selection will lead to inconsistent estimations. However, focusing on the variation in labour market outcomes between seemingly identical individuals that attended the same university department in different moments of time, it is possible both to relax the selection-on-observables assumption and to control for the existence of time-invariant differences in department unobservable characteristics:
L = α + Xβ + Rtfu γ + Utfu δ + Dd η + Dt ζ + Dfu µ +
(4)
where Dfu – is a set of dummies for each university department in the country. Model (4), instead of relying on the orthogonality of university- and individual-specific unobservables, is build on the assumption of time-stability of the parameters. The potential problem of this specification could arise from the limited variability of academic research activity in the short period of 6 years covered by our dataset. When using the number of citations to capture academic research quality Rtfu in model (4), one need to account for the truncation problem in the citations of more recent years. Normalization, described in Section 3, helps to solve this problem in our case.11 If research and teaching are positively correlated, the human capital hypothesis would suggest that graduates from a department performing a high-quality research activity are 11
Note that inclusion of non-normalized values of the number of publications (patents) and their citations into Rtfu presumes the equality of the marginal effects of one additional publication (patent) on teaching quality across different disciplines. This is a very questionable assumption given big differences across disciplines in professors’ propensities to patent and to cite other patents. Instead the use of the normalized research indicators presumes that a one standard deviation improvement in the academic research activity as compared to the average of the corresponding discipline produces similar effect on teaching quality across all disciplines.
17
likely to accumulate relatively higher human capital and thus to perform better in the labour market. However, the positive correlation between academic research quality and graduates’ performance in the labour market can be also interpreted in several other ways. First of all, good academic researchers are likely to be part of various research and innovation networks. Students of these academics might have easier access to information about the job offers available at network organizations. The recommendations of students’ professors might additionally facilitate the job search. In our database we have information on the channel, using which a student has found her current job and we know whether this occurred via the notice for employers done by the university, training centers or professors. We test whether graduating from a university department with better academic researchers is also associated with higher probability to find a job using this channel. Finally, academic research might have a direct impact on the probability that a student is going to choose a research-oriented career. We analyze the effect of academic research quality at the department on the probability to enroll into a PhD program after graduation. Given that the opening of a PhD program at the department can affect both the quality of the academic staff and the proportion of students at the department who decide to do a PhD12 , we control for the number of PhD scholarships available at the department. We also analyze, whether graduates from a relatively better research department are more likely to get a job as a researcher in a private sector.
4.2
Graduates’ satisfaction with teaching
We first exploit the cross-sectional data for 1995 to analyze how students’ evaluation of professors’ competence in the subject of their teaching depends on the quality of research carried out by these professors. The first look on the unconditional relationship between expert evaluation scores and students’ satisfaction with professor competence suggests that 12
In Italy many graduates who decide to do a PhD enroll into the PhD program of the same university where they completed their undergraduate studies.
18
indeed there is an apparent positive relationship between the two (see Table 7 column 1).13 Figures 1-4 visualize this result for selected disciplines. As column 2 suggests, most of this relationship is due to the regional differences: the inclusion of the dummies for Italian macroregions decreases the estimate twofolds. The coefficient estimated for the relationship between expert evaluation scores and professor competence in column 1 of Table 7 is biased if students of different quality or different socioeconomic background are more likely to have different satisfaction rates with respect to professor competence and at the same time are likely to self-select into institutions with different research intensity. Column 3 reports the same coefficient estimated conditional on a set of individual characteristics.14 We can observe that the coefficient has reduced substantially after conditioning on observed individual characteristics but remained significant at 1 percent level. Potentially, there exist several department characteristics that might effect both the department research output and the quality of teaching. Failure to control for these characteristics might lead to an inconsistent estimation of teaching-research relationship. In column 4 we control for those department characteristics that we thought could affect both teaching and research outputs. These characteristics include the indicator for private universities, student-per-professor ratio, the (log) number of professors at the department and mean professor age.15 All the department-level characteristics have an expected effect on the quality of teaching. Graduates from private universities are on average more satisfied with their professors than graduates from public institutions. Higher student-per-professor ratios lead to a lower student satisfaction. Finally, moving from small to bigger departments, graduates tend to be more satisfied with teaching. 13
Note that all standard errors are corrected for the potential clustering of the residual at the universitytimes-scientific area level. In order to facilitate the interpretation and the exposition of the results, this paper presents OLS estimates for all analysis that follows. Given the discrete nature of student satisfaction measures, we also estimated the corresponding models using ordered probit. The results are qualitatively similar and available upon request. 14 The estimates of individual characteristics effect on academic and labour market performance could be found in Bagues, Sylos Labini, Zinovyeva (2008) . 15 Result is not shown but available upon request.
19
Inclusion of the department characteristics into regression decreases the correlation coefficient by about 25 percent. Though, it remains significant at 5 percent level. Our department-level controls might still not capture important dimensions of university quality that might be correlated with both student satisfaction with teaching and the quality of academic research. We thus proceed including university fixed effects among the covariates (column 5). The inclusion of these controls limits substantially the variation in academic research quality that we exploit for identifying its effect on students’ satisfaction with teaching. In fact, the estimated coefficient for CIVR score, though remaining positive, drops substantially and becomes insignificant. We then repeat the above estimations of students’ satisfaction with professor competence using, respectively, publication (publication citation)-per-professor and patent (patent citation)-per-professor ratios as the measures of academic research output. This information is only available for departments in sciences, medicine and engineering. So the estimations which rely on these measures of academic research quality refer only to graduates from these departments. We also check whether the relationship between expert evaluation scores and students’ evaluations of teaching for graduates in these disciplines is different from the aggregate one. As before, we find that our measures of research output of science, medicine and engineering departments are highly correlated with student satisfaction even after controlling for a number of individual and department-level characteristics (see columns 6-9). Moreover, the positive correlation between students’ satisfaction with teaching and patenting activity of professors remains significant after controlling for university fixed effects (see columns 10). The above evidence suggest that students correctly perceive how competent or knowledgeable their professors are. The next question would be weather students are actually more satisfied with the clarity of explanations if they attend a degree in a department with better researchers. Tables 8 summarizes corresponding estimation results. Columns 1-5 show that on aggregate the estimates for the relationship between academic research and stu-
20
dents’ satisfaction with the clarity of professors’ teaching are very similar to the ones for the corresponding relationship with students’ evaluation of professor competence. Though, the estimates in the former case are a bit less precise, especially on the subsample of graduates in sciences, medicine and engineering. If we interpret our results as causal, the research activity of universities increases student satisfaction with professor competence. It also somewhat increases the clarity of teaching as perceived by students. Obviously, there might be other channels through which professors transmit their knowledge to students. For example, better researchers might adjust the course content making it more up-to-date and more relevant for applications.
4.3
Graduates’ labour market performance
If research and teaching activities are complementary, the human capital theory would predict that students from relatively better research departments perform better in the labour market after graduation. Thus labour market outcomes of graduates could be considered as an objective measure of university teaching quality. Moreover, departments with better academic research might provide an additional reputation signal for their graduates in the labour market. We look at three main indicators of labour market performance: probability to be employed 3 years after graduation conditional on being in the labour force, earnings if employed, and the type of career undertaken or, more precisely, the probability to undertake a researchoriented career. In Table 9 we analyze the effect of academic research carried on at the department on the probability of an individual to be occupied 3 years after graduation (conditional on being in the labour force). Note that this indicator gives us an idea of students performance only in the short run. In all specifications except from the ones of columns 1 and 5 we control for the fact that a graduate undertook some post-graduate education, but still the interpretation of results should be considered with caution. 21
Unconditional correlations in columns 1 and 6 suggest that on average graduates from better research departments are more likely to be employed 3 years after graduation. Still, most of this correlation is across regions and not within regions, as columns 2 and 7 suggest. However, our results suggest that publishing record of the science, medicine and engineering departments is correlated with graduates occupation probability even after controlling for a number of department-level characteristics and university fixed-effects (columns 9 and 10). The positive correlation between publication intensity and graduates’ occupation is observed also if we exploit only over-time variation of both outcomes of each department (column 11). This positive relationship is also observed if we measure the quality of academic research by the number of citations received by academic publications or academic patents. Table 10 shows the estimation results for the effect of research activity on graduates’ future wage. Note that in the analysis presented in Table 10 we control for the province of birth and also for the region of work. This exclude the possibility that our results are biased by the differences in purchasing parity and living standards across the regions of Italy. Once again, we find that the departments that manage to produce highly cited patents are also better in terms of graduates’ earnings. The result is robust to any observable individual, university or department control. Finally, we consider the effect of department academic research on the probability that its graduates undertake a research career in the future. More specifically, we analyze whether graduates from institutions that display better academic research records are more likely to be enroled in PhD programs or employed as researchers three years after graduation. Columns 1-5 of Table 11 report the results obtained using the entire sample of graduates (i.e. including humanities and social sciences degrees). We observe the positive correlation between the CIVR score and the likelihood of undertaking a research-oriented career which is robust to controlling for individual characteristics, observed department characteristics and university fixed effects. Note that in Italy most doctoral programs are endowed with publicly funded scholarships
22
assigned at the university level. For example, according to official data, in 2001 about 60 per cent of all PhD students were recipients of one of these scholarships. Given the low mobility of Italian graduate students (i.e. they often enrol in a PhD offered by the same university where they graduated), our results could be potentially driven by scholarship availability at the department level. The latter is in fact likely to be correlated with both the research performance of a given institution and the likelihood that internal graduates enrol in a doctoral program. So, although this control is potentially endogenous, we include among department controls the number of PhD scholarship per graduate granted at the department level. A causal interpretation would suggest that a one-standard-deviation increase in the CIVR score increases the probability of undertaking a research career by about 0.4 percentage points. The effect of research quality almost doubles for graduates in science, medicine, and engineering: a one-standard-deviation increase in the CIVR score augments the likelihood of becoming a researcher by 0.8 percentage points. This is a relatively high effect, taking into account that only around 10 per cent of graduates in science, medicine and engineering become either researchers or PhD students. Similar results could be observed if instead of CIVR score we use publication or publication citations per professor as a measure of research quality. Still, exploiting variation in department publication activity over time does now allow us to confirm our cross-sectional finding. Estimates for patent-related measures remain not statistically different from zero as well.
5
Conclusions
In this paper we analyzed the effect of research activity at the department level on teaching quality and labour market performance. Our results suggest that overall graduates from departments with better academic re-
23
search quality are relatively more satisfied with professors competence and the clarity of their lectures. We use several measure of graduates’ labour market performance as alternative objective measures of graduates’ preparation. We find evidence of a positive relationship between graduates performance and academic research. Specifically, we find that more science-oriented academic research (as measured by expert evaluations, publication record or patent citations) increases graduates’ probability to be employed 3 years after graduation. Additionally, rather applied academic research (as measured by patenting activity) seems to be associated with higher earnings of graduates. Moreover, scientific research motivates graduates into choosing research-oriented careers. To sum it up, our analysis suggests that Italian universities producing better research output are also the ones whose graduates receive better training. Therefore, allocating public funds to universities according to their research productivity does not seem a bad idea. However, institutions, likewise individuals, respond to incentives and, thus, we do not know whether this correlation would persist if more financial incentives to research quality will not be balanced by a more careful assessment of teaching quality.
References Bagues, M., M. Sylos Labini, and N. Zinovyeva (2008). Differential grading standards and university funding: Evidence from Italy. CESifo Economic Studies, forthcoming. Becker, W. E. and P. E. Kennedy (2005). Does Teaching Enhance Research in Economics? The American Economic Review 95 (2)) Papers and Proceedings of the One Hundred Seventeenth Annual Meeting of the American Economic Association,Philadelphia, PA, January 7-9, 2005, 172–176. Belfield, C. and A. Fielding (2001). Measuring the relationship between resources and outcomes in higher education in the UK. Economics of Education Review 20(6), 589–
24
602. Black, D. A. and J. A. Smith (2004). How robust is the evidence on the effects of college quality? Evidence from matching. Journal of Econometrics (121), 99–124. Breno, E., G. Fava, V. Guardabasso, and M. Stefanelli (2002). La ricerca scientifica nelle universit italiane : una prima analisi delle citazioni della banca dati ISI. Roma CRUI . Brewer, D., E. Eide, and R. Ehrenberg (1999). Does it pay to attend an elite college? cross cohort evidence on the effects of college type on earnings. Journal of Human Resources 34(1), 104–123. Brunello, G. and L. Cappellari (2005). The labour market effects of alma mater: Evidence from Italy. IZA Discussion Paper (1562). Chevalier, A. and G. Conlon (2003). Does it pay to attend a prestigious university? CEP Discussion Paper, LSE . Dale, S. B. and A. Krueger (2002). Estimating the payoff to attending a more selective college: An application of selection on observables and unobservables. Quarterly Journal of Economics (117), 1491–1528. Di Pietro, G. and A. Cutillo (2006). University Quality and Labour Market Outcomes in Italy. Labour 20 (1) 3762. Dosi, G., P. Llerena and M. Sylos labini (2006). The relationships between science, technologies and their industrial exploitation: An illustration through the myths and realities of the so-called ’European Paradox’. Research Policy (35), 1450-1464. Gautier, A. and X. Wauthy (2007). Teaching versus research: A multi-tasking approach to multi-department universities. European Economic Review (51), 273–295. Hattie, J. and H. W. Marsh (1996). The relationship between research and teaching: A meta-analysis. Review of Educational Research 66 (4), 507–542. Heckman, J., A. Layne-Farrar, and P. Todd (1996). Human capital pricing equations with 25
an application to estimating the effect of school quality on earnings. The Review of Economics and Statistics (78), 562–610. James, E., N. Alsalam, J. Conaty, and D. To (1989). College quality and future earnings: Where should you send your child to college? American Economic Review 79 (2), 247– 252. King, D. A. (2004). The scientific impact of nations. Nature (430), 311–316. Lissoni, F., B. Sanditov, and G. Tarasconi (2006). The Keins Database on Academic Inventors: Methodology and Contents CESPRI Working Papers 181 . Marsh, H. W. and J. Hattie (2002). The relation between research productivity and teaching effectiveness: Complementary, ANtagonistic, or independent constructs? The Journal of Higher Education 73 (5), 603–641. Mas-Colell, A. (2003, November-December). The European space of higher education: Incentive and governance issues. Rivista di Politica Economica, 9–27. McGuinness, S. (2003). University quality and labour market outcomes. Applied Economics 35(18), pages 1943-1955. Mowery, D. C. and N. Rosenberg (1989). Technology and the Pursuit of Economic Growth. New York: Cambridge University Press. Mowery, D. C. and N. Rosenberg (1993). The U.S. National Innovation System. (National Systems of Innovation: A Comparative Analysis. ed.). New York: Oxford University Press. Patrick and Stanley (1998). Teaching and Research Quality Indicators and the Shaping of Higher Education. Research in Higher Education 39 (1), 19–41. Perotti, R. (2002, January). The Italian university system: Rules versus incentives. Urwin, P. and G. Di Pietro (2005). The Impact of Research and Teaching Quality Inputs on the Employment Outcomes of Postgraduates. Higher Education Quarterly 59 (4), 26
275–295.
27
Table 1: Top Research Institutions, by country Name
Number of HCRs United States
Harvard University Stanford University National Institute of Health University of California, Berkeley Massachusetts Institute of Technology United Kingdom University of Cambridge University of Oxford Imperial College London University College London King’s College London
220 153 105 88 86 53 47 38 34 17
Germany Max Planck Institute Technische Universit¨ at M¨ unchen Universit¨ at W¨ urzburg Universit¨ at Hamburg Deutsches Krebsforschungszentrum
84 8 7 6 6
France National Institute for Scientific Research (CNRS) Universit´ e Pierre et Marie Curie National Institute for Agricultural Research (INRA) Institut Pasteur Universit´ e Louis Pasteur
15 10 8 6 5
Italy National Institute of Astrophysics (INAF) University of Milan University of Pisa University of Florence Menarini Ricerche S.p.A.
10 6 5 5 4
Spain National Institute of Scientific Research (CSIC) Universidad Aut´ onoma de Madrid Almirall Prodesfarma Research Center Universidad Polit´ ecnica de Valencia Universitat Pompeu Fabra
6 4 2 1 1
Notes: HCRs are top cited researchers in different fields of science for the period 1981-1999. See http://isihighlycited.com/ for details. Researchers highly cited in more than one field have been counted several times.
28
Table 2: Descriptive Statistics: Individual Characteristics Mean
Min
Max
0.532
0
1
27.587
21
75
- working
0.960
0
1
- looking for a job
0.004
0
1
- a pensioner
0.017
0
1
- other
0.019
0
1
- working
0.494
0
1
- looking for a job
0.004
0
1
- a pensioner
0.020
0
1
- other When an individual was 14 years old his father’s
0.482
0
1
- elementary license or none
0.190
0
1
- secondary education license
0.236
0
1
- higher education diploma
0.340
0
1
- university degree
0.226
0
1
- no answer When an individual was 14 years old his mother’s
0.008
0
1
- elementary license or none
0.250
0
1
- secondary education license
0.259
0
1
1. Predetermined Individual Characteristics Gender (Share of females) Age When an individual was 14 years old his father was:
When an individual was 14 years old his mother was:
highest educational title was:
highest educational title was:
- higher education diploma
0.350
0
1
- university degree
0.135
0
1
- no answer
0.006
0
1
- agriculture
0.050
0
1
- industry
0.260
0
1
- services
0.672
0
1
- no answer
0.018
0
1
- scientific lyceum
0.413
0
1
- classic lyceum
0.193
0
1
- technical industrial institute
0.062
0
1
- technical institute for geometers
0.034
0
1
- technical commercial institute
0.128
0
1
- other type of technical institute
0.030
0
1
Father’s sector of work
Type of high school:
(continued)
29
Table 2: (continued) Mean
Min
Max
- teachers school or institute
0.062
0
1
- language lyceum
0.036
0
1
- professional institute
0.029
0
1
- art lyceum or institute
0.013
0
1
49.085
36
60
2
0
4
103.628
66
110
In the labour force
0.843
0
1
Employed
0.736
0
1
Employed if in the labour force
0.853
0
1
1135.786
77.468
10000
0.064
0
1
High school grade 2. College-related individual characteristics Median number of extra years taken to graduate after the end of the official program duration (4 stands for 4 and more years) University grade 3. Graduates’ Post-Graduation Performance
Wage Researcher or PhD student
Notes:The number of observations is 61844. (*) In this case the median value is reported instead of the mean. Value 4 means that 4 or more extra years to graduate have been employed. (**) The number of observations with non-missing wage is 37552.
Table 3: Descriptive Statistics: Department and University Characteristics
Private universities Number of departments in university Number of professors Age of professors Student per professor ratio PhD scholarships per graduate
Mean
Std. Dev.
Min
Max
.089 7.446 165.575 49.712 32.398 0.053
.284 2.906 184.583 3.438 36.128 0.061
0 1 4 28 2.773 0
1 12 2073 72 408 0.500
30
Table 4: Descriptive Statistics: Academic Research Indicators
CIVR Publications Citations Patents Patent Citations
1995
1998
2001
0.786 (0.097) 124.424 (141.981) 1058.444 (1665.5) 0.642 (1.134) 1.050 (2.080)
0.781 (0.098) 147.908 (180.734) 301.305 (516.488) 0.915 (1.563) 0.908 (2.194)
0.771 (0.101) 153.245 (209.470) 2233.779 (3952.248) 0.867 (1.380) 0.232 (0.508)
Table 5: Correlations between research indicators Publications per professor
CIVR score CIVR score
Patents per professor
Patent citations per professor
1
Publications per professor Citations professor
Citations per professor
0.278 (0.000)
1
0.321 (0.000)
0.861 (0.000)
1
0.200 (0.000)
0.177 (0.000)
0.141 (0.000)
1
0.105 (0.000)
0.216 (0.000)
0.188 (0.000)
0.623 (0.000)
per
Patents per professor Patent citations per professor
1
Notes: P-values in parentheses.
Table 6: Data availability for academic research indicators, student evaluations and graduates labour market outcomes 1992 Surveys of graduates Evaluations of teaching Labor market outcomes Research variables Expert evaluation score Publications Patents
1993
1994
1995
1996
1997
+ + +
1998
1999
2000
2001
+
+
+
+
2001-2003
+ +
+
+
+ +
+ +
Note: + indicates availability of corresponding information.
31
+ +
+ +
+ +
+
+ +
Table 7: Academic Research and Student Satisfaction with Professor Competence 1
2
3
4
5
6
All disciplines CIVR score
0.048*** (0.011)
0.025** (0.012)
7
8
9
10
Sciences, Medicine and Engineering
0.025** (0.010)
0.018** (0.009)
0.008 (0.008)
0.061*** (0.010)
0.039*** (0.011)
0.043*** (0.012)
0.039*** (0.012)
0.024* (0.013)
Publications per professor
n/a
n/a
n/a
n/a
0.055*** (0.012)
0.027** (0.011)
0.031*** (0.012)
0.037*** (0.012)
0.015 (0.012)
Citations per professor
n/a
n/a
n/a
n/a
Patents per professor
n/a
n/a
n/a
n/a
0.046*** (0.011) 0.071*** (0.015)
0.018* (0.011) 0.042*** (0.012)
0.022** (0.011) 0.045*** (0.012)
0.027** (0.011) 0.041*** (0.011)
0.006 (0.011) 0.060*** (0.013)
Patent citations per professor
n/a
n/a
n/a
n/a
0.067*** (0.014)
0.039*** (0.011)
0.040*** (0.012)
0.037*** (0.011)
0.042*** (0.011)
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
Controls: Year Course dummies University marcoregion Individual characteristics University characteristics University dummies Department dummies
+ +
+ + +
+
-
-
+
+
+
-
-
+
+
+
-
-
-
+ -
+ + -
-
-
-
+ -
+ + -
Notes: Number of observations: 14503 in columns 1-5, 7147 in columns 6-10. In parentheses the standard errors clustered for graduates from the same university and discipline. * p-value<0.100, ** p-value<0.050, *** p-value<0.010. Individual characteristics controls include gender, age groups, type of the high-school, high-school grade, parents’ educational and occupational status, father’s sector of work, province of origin. Department characteristics include a dummy for private universities, student-per-professor ratio, log number of professors, average professor age.
32
Table 8: Academic Research and Student Satisfaction with Professor Clarity of Explanations 1
2
3
4
5
6
All disciplines CIVR score
0.031** (0.012)
0.026** (0.013)
7
8
9
10
Sciences, Medicine and Engineering
0.027** (0.012)
0.020* (0.011)
0.008 (0.010)
0.028** (0.012)
0.025** (0.012)
0.043*** (0.012)
0.039*** (0.012)
0.024* (0.013)
Publications per professor
n/a
n/a
n/a
n/a
0.005 (0.013)
-0.002 (0.013)
-0.002 (0.012)
0.001 (0.013)
-0.004 (0.016)
Citations per professor
n/a
n/a
n/a
n/a
Patents per professor
n/a
n/a
n/a
n/a
0.005 (0.013) 0.038*** (0.012)
-0.002 (0.013) 0.032** (0.013)
-0.001 (0.012) 0.026* (0.013)
0.002 (0.013) 0.026* (0.014)
-0.003 (0.015) 0.025 (0.017)
Patent citations per professor
n/a
n/a
n/a
n/a
0.029** (0.012)
0.024* (0.014)
0.014 (0.014)
0.015 (0.014)
0.015 (0.016)
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
Controls: Year Course dummies University marcoregion Individual characteristics University characteristics University dummies Department dummies
+ +
+ + +
+
-
-
+
+
+
-
-
+
+
+
-
-
-
+ -
+ + -
-
-
-
+ -
+ + -
Number of observations: 14372 in columns 1-5, 7102 in columns 6-10. Notes: In parentheses the standard errors clustered for graduates from the same university and discipline. * p-value<0.100, ** p-value<0.050, *** p-value<0.010. Individual characteristics controls include gender, age groups, type of the high-school, high-school grade, parents’ educational and occupational status, father’s sector of work, province of origin. Department characteristics include a dummy for private universities, student-per-professor ratio, log number of professors, average professor age.
33
Table 9: Academic Research and Graduates’ Employment Probability 1
2
3
4
5
6
7
All disciplines CIVR score
10
11
0.007** (0.004)
0.002 (0.003)
0.028*** (0.005)
0.002 (0.005)
0.007* (0.004)
0.008* (0.004)
0.005 (0.004)
-
n/a
n/a
n/a
n/a
0.030*** (0.005)
0.012*** (0.004)
0.013*** (0.004)
0.015*** (0.004)
0.012*** (0.005)
0.014* (0.007)
Citations per professor
n/a
n/a
n/a
n/a
Patents per professor
n/a
n/a
n/a
n/a
0.029*** (0.005) 0.015*** (0.005)
0.011*** (0.004) -0.003 (0.005)
0.013*** (0.004) -0.004 (0.005)
0.014*** (0.004) -0.005 (0.005)
0.011*** (0.004) -0.006 (0.005)
0.011** (0.005) -0.005 (0.006)
Patent citations per professor
n/a
n/a
n/a
n/a
0.017*** (0.006)
0.008 (0.005)
0.008* (0.005)
0.008* (0.005)
0.008 (0.005)
0.011* (0.006)
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
Controls: Year Course dummies University marcoregion Individual characteristics University characteristics University dummies Department dummies
0.006* (0.004)
9
0.009** (0.003)
Publications per professor
0.047*** (0.006)
8
Sciences, Medicine and Engineering
+ +
+ +
-
-
+
+
+
-
-
+
+
+
+
-
-
-
+ -
+ + -
-
-
-
+ -
+ + -
+
+
+
+
Number of observations: 51690 in columns 1-5, 23523 in columns 6-11. Notes: In parentheses the standard errors clustered for graduates from the same university and discipline. * p-value<0.100, ** p-value<0.050, *** p-value<0.010. Individual characteristics controls include gender, age groups, type of the high-school, high-school grade, parents’ educational and occupational status, father’s sector of work, province of origin, indicators of whether an individual is in graduate education, concluded or interrupted graduate education. Department characteristics include a dummy for private universities, student-per-professor ratio, log number of professors, average professor age.
34
Table 10: Academic Research and Graduates’ Earnings 1
2
3
4
5
6
7
All disciplines CIVR score
0.043*** (0.006)
0.012*** (0.005)
8
9
10
11
Sciences, Medicine and Engineering
0.009* (0.005)
0.006 (0.005)
0.001 (0.004)
0.035*** (0.008)
0.013* (0.008)
0.007 (0.007)
0.005 (0.007)
0.004 (0.007)
-
Publications per professor
n/a
n/a
n/a
n/a
0.020** (0.009)
0.004 (0.007)
0.001 (0.007)
0.006 (0.007)
-0.006 (0.007)
-0.017 (0.010)
Citations per professor
n/a
n/a
n/a
n/a
Patents per professor
n/a
n/a
n/a
n/a
0.016** (0.008) 0.026*** (0.007)
0.002 (0.007) 0.014** (0.006)
0.000 (0.006) 0.011** (0.005)
0.004 (0.006) 0.010* (0.005)
-0.006 (0.006) 0.008 (0.005)
-0.017 (0.010) 0.009 (0.008)
Patent citations per professor
n/a
n/a
n/a
n/a
0.025*** (0.007)
0.019*** (0.005)
0.016*** (0.005)
0.016*** (0.005)
0.014*** (0.004)
0.027*** (0.006)
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
Controls: Year Course dummies University marcoregion Individual characteristics University characteristics University dummies Department dummies
+ +
+ + +
+
-
-
+
+
+
-
-
+
+
+
+
-
-
-
+ -
+ + -
-
-
-
+ -
+ + -
+
+
Number of observations: 36917 in columns 1-5, 17195 in columns 6-11. Notes: OLS estimates. In parentheses the standard errors clustered for graduates from the same university and discipline. * p-value<0.100, ** p-value<0.050, *** p-value<0.010. Individual characteristics controls include gender, age groups, type of the high-school, high-school grade, parents’ educational and occupational status, father’s sector of work, province of origin, indicators of whether an individual is in graduate education, concluded or interrupted graduate education. Department characteristics include a dummy for private universities, student-per-professor ratio, log number of professors, average professor age.
35
Table 11: Academic Research and the Probability to Do a Research-Oriented Career, by discipline 1
2
3
4
5
6
7
All disciplines CIVR score
10
11
0.005*** (0.002)
0.004*** (0.002)
0.005 (0.003)
0.007** (0.003)
0.007** (0.003)
0.010*** (0.003)
0.010*** (0.004)
-
n/a
n/a
n/a
n/a
0.010*** (0.004)
0.011*** (0.004)
0.012*** (0.004)
0.010** (0.004)
0.013*** (0.005)
0.005 (0.009)
Citations per professor
n/a
n/a
n/a
n/a
Patents per professor
n/a
n/a
n/a
n/a
0.008** (0.003) 0.001 (0.006)
0.008** (0.003) 0.002 (0.006)
0.009*** (0.003) 0.001 (0.005)
0.008** (0.003) 0.004 (0.005)
0.009** (0.004) 0.003 (0.005)
-0.001 (0.007) 0.002 (0.007)
Patent citations per professor
n/a
n/a
n/a
n/a
0.005 (0.004)
0.006 (0.004)
0.005 (0.004)
0.006 (0.004)
0.004 (0.004)
0.006 (0.006)
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
+ +
Controls: Year Course dummies University marcoregion Individual characteristics University characteristics University dummies Department dummies
+ +
0.004*** (0.002)
9
0.005*** (0.001)
Publications per professor
0.004** (0.001)
8
Sciences, Medicine and Engineering
+ + +
+
-
-
+
+
+
-
-
+
+
+
+
-
-
-
+ -
+ + -
-
-
-
+ -
+ + -
+ +
Number of observations: 36482 in columns 1-5, 15980 in columns 6-11. Notes: OLS estimates. In parentheses the standard errors clustered for graduates from the same university and discipline. * p-value<0.100, ** p-value<0.050, *** p-value<0.010. Individual characteristics controls include gender, age groups, type of the high-school, high-school grade, parents’ educational and occupational status, father’s sector of work, province of origin, region of actual residence. Department characteristics include a dummy for private universities, student-per-professor ratio, log number of professors, average professor age, PhD scholarships per graduate.
36
2.8 3 3.23.4 3.6 3.8
3.5
3
2.5
2
2.62.8 3 3.2 3.4 3.6
2.5 3 3.5 4
-2
Palemo
-1
-3
Napoli
-1.5
0
Bologna
Catania
-1 0 VTR score
Camerino
2
1
2
Modena Genova Parma Pisa Torino PolitBologna L'Aquila MilanoPadova Polit
Cagliari
Perugina PadovaPavia
Law
-.5 0 VTR score
Pisa
.5
1
Palemo Napoli Catania Salerno Bari Polit Firenze RomaPavia La Sapienza Trento Cagliari Udine Trieste Brescia
Cassino
Engineering
LUISS
1
Trieste ParmaSiena Modena Pavia Cosenza Torino Ferrara Genova Venezia Lecce PisaMilano Roma La Sapienza Perugina Padova Napoli Bari Roma Tor Vergata Sassari Messina Firenze Urbino
Biosciences
VTR score
Milano
Udine Pisa
-2
0
VTR score
-1
1
2
Trieste Genova Catania Bologna Torino Sacro Cuore TrentoModena Bari Sassari Parma Catanzaro SienaFerrara Milano Messina Teramo MacerataCagliari Roma Tor Roma Vergata La Sapienza Campobasso Palemo Urbino Salerno Firenze Camerino
-1
Perugina
Ancona
Cosenza
Ferrara Firenze Camerino
Catania
Salerno Cosenza
Bari
Lecce
Roma Tor Vergata
-3
-2
Messina
Pavia Potenza Padova Roma Tor Vergata SacroRoma Cuore Bologna La Sapienza
Modena Napoli Torino L'Aquila
Trento Palemo Genova
Parma
Mathematics and Informatics
-2
0
Ferrara
Bari
Literature
-1
VTR score
Messina
0
1
Modena Catania
Bologna
Firenze
1
Chieti
Pavia RomaTorino Tor Vergata PieUdine Orien Milano Trieste Padova Ancona Pisa Cagliari Perugina Modena Napoli II Genova Verona Parma Brescia Roma Siena La Sapienza SacroBologna Cuore Firenze Napoli L'Aquila Sassari
Medicine
-1
VTR score
Messina
Trento
Torino Napoli Roma La Sapienza Roma Tor Padova Vergata Pisa Milano
Genova
-2
-1 VTR score 0
1
-1
0
VTR score
1
Trieste Bari Pisa Palemo Perugina Napoli Padova Cagliari Catania Messina Milano Teramo Sassari Trento Torino Roma La Sapienza Urbino Salerno
2
Pie Orien Macerata Siena Sacro Cuore FirenzePaviaNapoli Orient Bologna Genova
Political Sciences and Sociology
-2
Salerno
LUISS
Pavia L'Aquila Trento Viterbo Napoli Bergamo Roma Tor Vergata TriesteOrient Sacro Cuore Firenze Venezia Verona Potenza Padova Palemo Cosenza Pisa Torino UdineRoma Tre Genova Bologna Bari Milano Chieti Cagliari Macerata Roma LaSiena Sapienza Parma Lecce Cassino PeruginaCatania Sassari Messina Napoli Urbino
-3
-2
LUMSA
Catania Palemo
-3
Parma
Physics
0
VTR score
-1
1
-2
Catania
-1
Messina
Bari
0
Padova
1
Milano Napoli
Firenze Perugina Potenza Parma Torino Pisa
Sacro Cuore
Bologna Sassari
Viterbo
VTR score
Reggio Calabria
Palemo
2
Udine
Roma Ferrara La Sapienza
Agriculture and Veterinary
-2
Messina
Genova Trieste ModenaTorino Sassari Parma Padova Cagliari Urbino Napoli Pavia Camerino Perugina Bari Catania Siena
-3
-2
Catania
-1VTR score 0
Urbino
1
2
Pisa Napoli Torino Ferrara Milano Firenze L'Aquila Padova Bari Siena Roma Tre Trieste Sassari Parma Roma Tor Vergata Bologna Verona Roma La Sapienza Cagliari Cassino Perugina Salerno Chieti Palemo Genova Lecce Messina Napoli Benincasa
Cosenza
Macerata Venezia Sacro Cuore
History Psychology Philosophy Pedagogy
-3
-3
Palemo
Pisa Insubria Milano Bologna Firenze
Chemistry and Pharmacy
-1
0
VTR score
Firenze
Roma La Sapienza
Pavia Cagliari Genova Bari
Parma
Pisa Ferrara
1
Urbino
Milano
LUISS
-1
-2
L'Aquila Roma La Sapienza
Economics
0VTR score 1
Firenze Reggio Calabria
2
Chieti
Napoli Catania Potenza Venezia Genova Arch Milano Perugina Torino Polit Polit
Milano Bocconi
3
Padova
-1
0
VTR score
1
2
Sacro Cuore Cosenza Venezia Roma Perugina Tor Vergata Urbino Siena Padova Torino Verona Udine Pisa Bergamo Bologna PaviaModena RomaTrieste La Sapienza Brescia Trento Chieti Parma Campobasso Cassino Ancona Salerno Firenze Palemo Napoli Bari Cagliari Lecce Catania Napoli Navale Messina
Genova Foggia
-2
Bari Polit
Ancona Salerno Cosenza
Cagliari
Pavia Brescia Parma Bologna Palemo
Pisa
2
Padova
Architecture and Civil Engineering
-2
Catania
Modena Bologna Palemo Cosenza
Geosciences
Siena Napoli Camerino
Torino
Figure 1: Student Satisfaction with Professor Competence (1995) and Standardized Expert Evaluation of Research (2001-2003)
2
3 3.2 3.4 3.6 3.8
2.4 2.6 2.8 3 3.2 4 3.5 3 2 2.5 3.5 2.5 3 1.5 2
2.4 2.6 2.8 3 3.2 3.4
2.42.62.8 3 3.2 3.4 1 1.5 2 2.5 3 3.5
2.6 2.8 3 3.2 3.43.6 2.6 2.8 3 3.2 3.4 3.6 4 2 2.5 3 3.5
ygogadg en Pyiry geh osep lcnoo iy itigcsa ryn o am cE n S liarh irlo m diP evfntrn iyC a eIgV hd soPenldo cadnh nnase csacs g ry e iecsiu en as y c te rtS c zat ca a iP u n rnse a rgicien a tenm iu te rle ich n soete p b y a o re llnectVcae iou tsc iire e to c c rvSnilsa ic o A m lac irn e o a c e iarruotnCh o sian rO s h N P B risbantiIatCeomTa5 L ilnosId z e a iw A iz too n e lira n g iry o o rmnizvbaic n 8 2 4 6 5 riz m a e u n tR g a P ilS p o e c o tie s a lbtnoOeie m .5 g a g sro e n ic te s q e g a h n u o s m S ro n le M a n c d riuTrcnas1 3 2 p e g isd .5 c v rb tiA e riIlie tM o la sn e A d ilo U h H .aG 4 3 2 1 0 e rC B rP L E iV 'in M V A N -GUCTBSRPLF.I
37
Appendix Table A-1: Description of the variables Variables Individual characteristics Age Female Father’s education Mother’s education Province of origin
High-school grade Type of the high school
Course dummies PhD student Researcher Employed Job-via-University
Department characteristics Private university Number of professors Mean professor age Student-professor ratio PhD scholarships per graduate Research variables CIVR score Publications Publication citations Patents
Patent citations
Description
Source
Dummy variables for each age group A dummy variable for female respondents Dummy variables father education (Secondary education, higher education diploma, university degree, no answer) Dummy variables mother education (Secondary education, higher education diploma, university degree, no answer) Dummy variables indicating the province where an individual resided at the age of 14 (103 Italian provinces and a dummy for foreign residence). Final high school grade Dummies for each type of the high school: scientific lyceum, classic lyceum, technical industrial institute, technical institute for geometers, technical commercial institute, other type of technical institute, techers school of institute, language lyceum, professional institute, art lyceum or institute Dummy variables indicating the exact degree course attended by an individual in the university, 82 categories. A dummy variable indicating that an individual is currently enrolled in a PhD program A dummy variable indicating that an individual is currently employed as a researcher. Unconditional on employment. A dummy variable indicating that an individual is currently employed A dummy variable indicating that an individual has found his current job using university or professor recommendations
Graduates’ Survey Graduates’ Survey Graduates’ Survey Graduates’ Survey
Graduates’ Survey Graduates’ Survey
Graduates’ Survey Graduates’ Survey Graduates’ Survey Graduates’ Survey Graduates’ Survey
A dummy variable for private universities Number of tenured professors at the level of university disciplinary area Average age of university professors at the level of university disciplinary area Number of non-delayed students over the number of professors at the department level Number of PhD scholarships available per the total number of graduates at the level of university disciplinary area
MIUR MIUR MIUR
A score assigned to each university disciplinary area by the external committee evaluating the quality of academic research Number of ISI Thompson publications done by professors from a given university and disciplinary area in the period 1995-1999. Number of ISI Thompson citations received by publications up to 2001. Number of all patent applications done by academic inventors from a given university and disciplinary area with the priority date between 1993 and 2001 Number of all citations received by academic patents from other EPO patents up to 2004.
CIVR
MIUR MIUR
ISI and CRUI ISI and CRUI EPO and KEINS
EPO and KEINS
Abbreviations: MIUR is the Italian Ministry of Education and Research. CIVR is the Committee for Evaluation of Research at the Ministry of Education and Research. CRUI is the Conference of Italian University Presidents. EPO is the European patent office. KEINS stands for the the European research project ”Knowledge-based Entrepreneurship, Innovation Networks and Systems”.
38
