(The following is a slightly edited version of an invited post appearing on The Inside Agenda Blog on TVO‘s web space.)

Canada retreats from Science

Canada is retreating from investment in science and engineering. Public letters (by 10 prominent physicists, 336 mathematicians, 49 leading researchers) have signaled alarms at changes to the NSERC Discovery Grants Program and the elimination of the Major Resources Support (MRS) and Research Tools and Instruments (RTI) programs. Investments in the training of the next generation of researchers through the Postdoctoral Fellowships Program have been slashed.
NSERC Postdoctoral Fellowship Investment over time
Without funds to operate laboratories, without funds for new tools, and without funds for young researchers, Canada’s science and engineering research enterprise faces disaster.

Mission Drift

The program cuts are not driven by a decrease in the budget to NSERC. The program cuts are instead the result of a transfer of funds away from people and discovery into new programs giving money to businesses, a transformation characterized by the recent report of the federal R&D panel as “mission drift.” The Engage Program, described by NSERC President Suzanne Fortier as spawning business-academy “first dates” provides an illustration. Consider the details of the program:

  • NSERC provides $25K of taxpayer funds to pay for a six-month research and development project between a university researcher and a company already involved in research and development.
  • The company is not required to invest any money on the project.
  • Any intellectual property developed by the project is owned by the company.
  • There is no direct return back to taxpayers, to the university researcher, or to the university on the investment.

The program description reports that “these grants are intended to foster the development of new research partnerships between an academic researcher and a company that have never collaborated together before.” However, the Engage Program does not appear to be producing robust collaborative partnerships. There have been cool anecdotes about ski goggles and fiber optic guitar pickups but insufficient reporting on the program as a whole. Recently, in response to an inquiry from the official opposition regarding the conversion rate of Engage grants into the more substantial Collaborative Research and Development (CRD) grants, NSERC reported:

“348 distinct researchers have received both Engage grants and Collaborative Research and Development grants since these programs have operated, and this without regard to the years or the order in time. This number represents 10.62% of the total number of grantees for these two programs.”

This is a confusing statement and does not accurately reveal how many Engage Grants matriculated to become CRD projects. NSERC President Fortier has written that

“Nearly a thousand Canadian companies have benefited from the Engage experience to date.”

This represents an investment of $25,000,000. From a program level perspective, and not just anecdotally, what was achieved?

NSERC Budget Changes

Despite announcements to the contrary by NSERC and Minister of State (Science and Technology) Gary Goodyear, the evidence shows that NSERC and the NRC (now described as a “business concierge”) are transfering funds away from “blue sky” basic research programs to Canadian businesses through programs like Engage.

Inadequate Consultation

Major changes in NSERC funding have often involved the research community through a long range plan (LRP) consultation. Long range plans for Subatomic Physics and Astronomy were recently completed; the LRP for Mathematics/Statistics is close to completion. The LRP consultation process activates a nationwide discussion by a community of researchers, contributes scientific input to the federal research investment strategy, and, in some cases, identifies opportunities for cost savings. The broad consultation of the LRP process respectfully empowers researchers to contribute to the policy discussions affecting them and, ultimately, all of Canada.

In contrast, there was no broad consultation in advance of the recent decisions to eliminate the Major Resource Support (MRS) and Research Tools and Instruments (RTI) programs. Shortly after University of Ottawa Chemistry Professor David Bryce’s letter and related public messages appeared, Minister Goodyear announced that these actions would only be a moratorium for one year as the government “seeks counsel” from the scientific community. Minister Goodyear’s remarks were reassuring but the terms of the RTI consultation have turned out to be much more narrow in scope. Instead of seeking creative input from the Canadian scientific community on how best to consolidate the “plethora of programs” and to “simplify the application process,” the consultation asks scientists and engineers to choose between Option 1 (rock) and Option 2 (hard place).

Canada was and can be a spectacular place for scientific and engineering studies. Canada had a research investment strategy that was once the “envy of the world.” Rapid policy changes with inadequate participation by the research community in the decision process threaten Canada’s long-term prosperity.

, , , ,

 

I received today the September 2012 Contact Newsletter (volume 36, number 4) from NSERC via email. The fourth item in the newsletter reads:

Postdoctoral Fellowships – no change to number of awards

Over the last ten years, the volume of applications to the NSERC PDF Program has doubled to about 1,300, impacting the workload of volunteer selection committee members. A change to the eligibility rules for the Postdoctoral Fellowships (PDF) Program was made to ensure that applicants’ and reviewers’ time was used productively.

The eligibility rules were changed to allow students to apply only once during the eligibility window. Please note that this change does not affect the budget for the PDF Program or the number of awards.

More information about the new policy is outlined in the Program Guide for Students and Fellows.

Over the last ten years, the faculty at Canada’s Universities has expanded by the addition of 2000 Canada Research Chairs and other strategic recruitment. This “brain gain” has had the desired effect: more highly qualified personnel are being produced by the system. Over the period 1999-2009, there has been an expansion in enrollment in degree granting programs (data extracted from 2010-2011 Tables):

  • Bachelor’s enrollment expanded 34% (Table 44)
  • Master’s enrollment expanded by 56% (Table 45)
  • Doctoral enrollment expanded by 70% (Table 46)

It does involve a lot of work for volunteers to assess postdoctoral fellowship applications. Instead of punishing the next generation of scientists by restricting the number of competitions they can enter to one, an alternate solution to the workload problem would be to correspondingly expand the size of the volunteer review committee. I volunteer to help with those assessments in mathematics. Other Canadian scientists and engineers could indicate their willingness to help with the assessments by politely contacting their program officers.

The Contact Newsletter item also reports that the budget and number of awards will not be changed. Note the substantial changes that have already occurred between 2010 and 2012.

,

 

The (false) headline conveys the sporting analog of NSERC’s new policy on Postdoctoral Fellowship Competitions:

Effective as of the 2013 competition, you can only apply once to the NSERC Postdoctoral Fellowships (PDF) Program; however, applicants whose first PDF application was submitted prior to the 2013 competition may submit a second application provided they are within the eligibility window.

What’s going on? Why would Canada choose to limit the pool of participants competing for advanced training opportunities in science and engineering? A recent letter to the Canadian Association of Postdoctoral Scholars by NSERC’s Director (Scholarships and Fellowships Division) Serge Villemure gives the following reasons:

In recent years, NSERC has seen a growing disparity between the number of applications submitted to the Postdoctoral Fellowships (PDF) program and the number of awards available. As a result, NSERC has decided to reduce the maximum number of applications an individual may submit in a lifetime to its PDF program from two to one.

This change to the eligibility rules will contribute to a better alignment between both the number of applications submitted and the awards available, thereby streamlining the application and review processes. Limiting the number of applications an individual may submit to the program will not impact the the current budget projections or the number of anticipated awards available.

The success rate for the postdoctoral fellowships competition in 2011 was 9.3% and in 2012 the rate was 7.8%. (The tables and visualizations are appended below.) Another strategy to confront the “growing disparity” is to invest more money into Canadian human capacity for research and development by expanding the number of awards. However, changes in NSERC policy over the past decade have transferred investment away from its mission supporting discovery and the training of highly qualified personnel into many new programs aimed at commercialization of research. Restricting Canada’s young scientists to one postdoctoral fellowship competition per lifetime has “better alignment” with the transfer of funds toward commercialization, but it is a bad policy change.

Funding support for graduate programs from Ontario (and likely from other provinces too?) is frequently limited to four years. This means that faculty and departments are under pressure to have their graduate students complete their PhD in four years. Unfortunately, many students do not meet this timeline. Funds to pay for extensions of PhD studies into a fifth and sometimes a sixth year must come from other sources and are often uncertain, conditional upon adequate progress, and may involve expanded teaching responsibilities. Graduate students know all this.

Consider the point of view of a graduate student. Suppose the key advances for the student’s thesis are completed during the summer between the third and fourth year of studies and the student starts writing the thesis during the Fall of the fourth year. The funding uncertainty for the fifth year motivates the student to want to finish the PhD in the fourth year. To maintain a career in science, the student needs to also spend that Fall preparing job and fellowship applications, a process that can take up a lot of time and mental energy. The student’s application materials (research statement, letters of recommendation, thesis abstract) will be not as strong as they would be if the thesis were entirely nailed down. Nevertheless, the funding uncertainty for the fifth motivates the student to submit postdoc applications in the fourth year.

What happens next? In this situation, students sometimes get a postdoc but more frequently don’t. When they don’t, they stay on for another year and often make substantial advances. Their science comes together during the fourth year and the summer thereafter. They have a working draft of their thesis at the start of the fifth year and can concentrate on applications. Instead of merely talking about the student’s potential, the letters of recommendation can reference accomplishments. Students who fail to land a postdoc offer in their fourth year often emerge as extremely strong candidates in the next year.

PhD students will soon be asking graduate advisers for advice: should I apply for an NSERC postdoc now or should I wait until next year? The right answer was both. Under the new policy, the answer is not clear. A certain outcome: some excellent candidates will be forbidden to enter the competition because they applied the year before.

NSERC’s new one-postdoc-competition-per-lifetime rule combined with the funding uncertainties around fifth and sixth year support are a lethal combination. The victim is Canada’s scientific research capacity.

 


I’ve set up a “hive” on BuzzData (an open social media platform for discussions around data) focused on NSERC. My view is that there is a need for respectful discussion about Canada’s research and development policy driven by transparent data. So far, there are four public Datarooms devoted to the following topics:

Others are welcome to join the hive.


NSERC Funded PDFs data (Thanks David Kent.)

  • Awards/Applicants (Year)
  • 250 / 1169 (08)
  • 254 / 1220 (09)
  • 286 / 1341 (10)
  • 133 / 1431 (11)
  • 98 / 1254 (12)

(Extracted from NSERC’s Scholarships and Fellowships Competition Results.)

The acronyms appearing in the tables are defined as follows:

  • CGS M/PGS M (one-year scholarship for the first or second year of graduate studies);
  • CGS D2/PGS D2 (two-year scholarship tenable during the first five years of doctoral studies);
  • CGS D3/PGS D3 (three-year scholarship tenable during the first five years of doctoral studies); and
  • PDF (two-year postdoctoral fellowship).

Visualizations by Brent Pym:

 

2012-08-21 Addendum (New visualizations by Brent Pym.)

Investments by governments to support research and development are crucial to economic prosperity, job creation, scientific advancement, and improvements to the future to be inherited by children. How should these investments be selected?

  • Merit review is a competitive process leveraging the expertise of a specially qualified panel to direct investments in research and development.
  • Earmarks are appropriations given to specific recipients or targeted areas, without competition, to satisfy the intent of government.

When viewed over the time scale on which benefits of research occur (decades), the merit review process is the better strategy. When viewed over the time scale of election cycles (years), governments often consider earmarks to be the better strategy. Visionary governments who followed the advice of scientists like Francis Bacon and Vannevar Bush built research systems that spawned the industrial revolution, the space and electronics industries, the internet, …. Canadian governments of the past who listened to John Charles Fields and Maurice Lamontaigne launched Canadian industries through the NRC and NSERC.

“The wealth of a nation once depended on its natural resources or the sheer size of its land, or its potential labour force; but now it is coming to depend more on its reservoirs of knowledge and its ability to organize and utilize them than on the older criteria.” –A Science Policy for Canada, Lamontagne Report, v. 3, (1976)

(Concerning the allocation of research funds) “It is folly to use as one’s guide in the selection of fundamental science the criterion of utility. Not because (scientists)… despise utility. But because. .. useful outcomes are best identified after the making of discoveries, rather than before.” — John Polanyi, Speech to the Canadian Society for the Weizmann Institute of Science, Toronto (1996-06-02)

“Faced with the admitted difficulty of managing the creative process, we are doubling our efforts to do so. Is this because science has failed to deliver, having given us nothing more than nuclear power, penicillin, space travel, genetic engineering, transistors, and superconductors? Or is it because governments everywhere regard as a reproach activities they cannot advantageously control? They felt that way about the marketplace for goods, but trillions of wasted dollars later, they have come to recognize the efficiency of this self-regulating system. Not so, however, with the marketplace for ideas.” — John Polanyi, Quoted in Martin Moskovits (ed.), Science and Society, the John C. Polanyi Nobel Lareates Lectures (1995)

Instead of listening to today’s voice along this lineage, John Polanyi, Canada’s current federal1 government listens to Dean Roger Martin.

“What makes a country prosperous is not investment in science and technology. It is businesses producing high paying jobs by having unique products and processes that a customer needs.” — Roger Martin, Canada will shrivel under business-school neglect, dean says , The Globe and Mail (2011-03-16)

Canada, like Texas, doesn’t shrivel.

The report of the expert panel, the “Jenkins Report”, advising the government on the effectiveness of federal support of business research and development recommended2 the creation of a new Industrial Research and Innovation Council (IRIC) and separately recommended3 that the NRC be ramified “into a constellation” of new R&D centres, a natural progression following the birth of the Tri-council. Rather than constellating the NRC, Budget 2012 does the opposite with plans for the role of the proposed IRIC to be carried out by a more centralized, business-focused, NRC:

…the Government will consider ways to better focus the National Research Council on demand-driven research, consistent with the recommendations
of the Expert Panel. – Budget 2012

The policy implementations of Budget 2012 for CIHR, NSERC, SSHRC will be announced today in a meeting to Vice Presidents [Research] of Canadian universities. Drifting along a decade-long trend, a transfer of funds away from research investment programs distributed through competitive merit review toward new programs aimed at business recipients without competitive review is anticipated. Canadian business innovation gaps will not be solved by cutting funds supporting academic research and education activities.

The wisest investment strategy of federal dollars should rest upon systems involving the best expertise and considerations of the disruptive impact of basic research.


Footnotes:

  1. The current government of Ontario is similar: after earmarking \$50M to the Perimeter Institute in 2011, \$42M was cut out of research in 2012.
  2. “Recommendation 1: Create an Industrial Research and Innovation Council (IRIC), with a clear business innovation mandate (including delivery of business-facing innovation programs, development of a business innovation talent strategy, and other duties over time), and enhance the impact of programs through consolidation and improved whole-of-government evaluation.”
  3. “Recommendation 4: Transform the institutes of the National Research Council (NRC) into a constellation of large-scale, sectoral collaborative R&D centres involving business, the university sector and the provinces, while transferring NRC public policy-related research activity to the appropriate federal agencies.”

Resources:

,

Fifteen1 faculty members from the Department of Mathematics at the University of Toronto submitted proposals to the 2012 NSERC Discovery Grants competition. Of these, one was a first time applicant (En), two (Ga, Ia) applied after a successful appeal of 2011 results, and one (Cd) was an appellant whose appeal was denied but could reapply because the 2011 award was for zero dollars. The first table below shows the 2012 results (in thousands of dollars per year) with 2010, 2011 award amounts for those researchers. The second table shows similar data for Toronto mathematicians in the 2011 competition, including the amounts for researchers (1d, 2d, 3d, 4d) whose appeals of 2011 results were rejected. The average for 2012 awards to Toronto mathematicians was 153% the average for 2011.

Average Grant Amount (Toronto Math)

  • 2006: \$27k/y
  • 2007: \$26.3k/y
  • 2008: \$26.5k/y
  • 2009: \$25k/y
  • 2010: \$25.3k/y
  • 2011: \$19.3k/y
  • 2012: \$29.5k/y

Instability Visualized

Viewing the results from the perspective of researchers in these competitions reveals instability in the Discovery Grants evaluation and appeals processes:

  • Imagine the experience of researcher 2d. This person had five years at \$42k/y, was cut to \$30k/y in 2009, and successfully appealed that outcome. The result of the appeal was a one year reinstatement of the previous grant level at \$42k/y and permission to reapply to the 2011 competition. In 2011, this researcher’s grant was hacked to \$18k/y so this person files another appeal. The 2011 appeal is rejected.
  • Contrast the experience of professors 4d and Ga. Both launched their Canadian research careers and entered the Discovery Grants competition for the first time in 2011. Ga’s appeal of the \$13k/y result from 2011 was successful and the 2012 competition led to a new result of \$30k/y for the next five years. 4d’s 2011 appeal was denied so this researcher is locked in for five years at \$11k/y. Which of these researchers is likely to have better HQP numbers at renewal time five years from now?
  • The experience of 2011 appellant Ia is also a bit strange. After a long run of celebrated research funded at the \$40k/y level, this researcher’s funding level was dropped in 2011 to \$15k/y. That outcome was successfully appealed and the 2012 outcome was \$35k/y.
  • The 2011 appeals by 1d, 2d, 3d, and 4d were all turned down so these researchers are locked in at relatively low funding levels for the next five years.
  • NSERC deviated from standard policy (a “pilot program”) in their handling of the 2011 appeals of Toronto mathematicians. Toronto’s appeals were evaluated by multiple appeals advisers while appeals from other universities were evaluated by one. There is evidence2 showing that Toronto appeals were denied even when one of the appeals advisers advocated for granting the appeal.

Consistent Results on Appeals Cases show 2011 was Anomalous

Proposals by three Toronto researchers were evaluated in both the 2011 and 2012 competitions. The outcomes for these proposals provide a comparison3 between the accuracy of the merit evaluations by the 2011 and 2012 Evaluation Groups and bin-to-funding assignment by the Executive Committee and NSERC staff. Here is the data, including the percentage adjustment from 2011 to 2012:

These three cases provide further evidence, consistent with the message in the public statement signed by over 300 Canadian researchers, that the 2011 evaluations were anomalous. Despite the consensus opinion from the Canadian math/stats community, a public message from a majority of the 2011 Evaluation Group, and advice from top administrators that the 2011 anomalies required an altered appeals process, NSERC chose not to reevaluate the scientific merit of proposals when considering whether to grant or deny an appeal. The grounds for a successful appeal required evidence of administrative errors; evidence of error in merit evaluation was not considered germane.

The Math-NSERC Liaison Committee is collecting data from department chairs about the 2012 competition for Section 1508. It might turn out that 2012 will be viewed as more consistent with expectations, a more accurate evaluation compared to 2011. This would be encouraging. However, the unsuccessful 2011 Toronto “pilot program” appellants (researchers 1d, 2d, 3d, 4d) face the next five years with inadequate funding to support their research programs.


Footnotes:

  1. The names of the faculty members are suppressed. The 2012 applicants will be referenced with codes A, B, C, …, H; 2011 applicants (without successful appeal) will be referenced using 1, 2, …, 7. The appended small case letters indicate whether the researcher had an appeal granted (a), an appeal denied (d), or was a new applicant (n).
  2. This evidence is contained in the reports of the appeals advisers provided to the appellants by NSERC and also in documents obtained by some of the appellants through formal requests under the Access to Information Act.
  3. It would be useful to know the data (number, success rate, basis for granting) for Discovery Grant appeals submitted to NSERC over the past few years. As far as I can tell, NSERC does not provide this data.

 

, ,