LST has received a number of inquiries from schools since updating our employment data clearinghouse. In most instances the schools did not understand the data they were publishing, either on their websites or through U.S. News.

Of the inquiries we received, two complaints, which came from administrators at Santa Clara and Toledo, warranted updates/corrections. These schools informed us that we were using wrong or incomplete data. They were right in this regard, though the problems stemmed exclusively from what schools supplied to U.S. News.

In Toledo’s case, a law school representative misinterpreted questions on the U.S. News survey and therefore supplied the magazine with incorrect data. In Santa Clara’s case, the school had policies and procedures in place that led to under-reporting what the school actually knew about its graduates. In both cases we were able to work with the schools to identify the source of the problems and have corrected the errors with data supplied by the schools.

(We’d also like to mention that prior to releasing the class of 2010 data clearinghouse, we
contacted ten law schools that made the same mistake as Toledo with their U.S. News-supplied data. Four of these schools confirmed the discrepancies and have provided us with the correct data.)

We’ve added the following note to Santa Clara’s class of 2010 profile:

After joint review with Santa Clara, we have restored the school’s profile using data provided by Santa Clara following its internal review of each 2010 graduate’s student file. Rather than relying on student-supplied data, which is what the school reported to U.S. News and reported on its website (the original data in the school’s profile), Santa Clara added data the administration culled from conversations and basic investigation.

Note: One major change is with the 28 jobs Santa Clara originally reported as non-professional. Santa Clara tells us “[t]his was done in error.” While these graduates were still employed, Santa Clara does not know what sort of credentials (e.g. bar passage required) those graduates’ jobs required. However, Santa Clara does know 12 of these 28 graduates’ employer types (e.g. law firm) and expected working hours (i.e. FT or PT).

We are happy to report that Toledo chose to resolve questions about graduate employment outcomes by disclosing the 2010 NALP report, joining 47 other ABA-approved law schools who have done so. Toledo’s class of 2010 NALP report now appears in our report database and can be viewed here.

Finally, we welcome law schools to continue contacting us with their concerns. These conversations are always valuable and almost always lead to improving law school transparency.

New York University School of Law has decided to continue withholding valuable employment data from prospective students. After sending NYU a third request to publish its NALP report, we finally received a reply from Dean Ricky Revesz:

Thank you for your note. I expect you are aware that, since the end of last year, we have added a substantial amount of employment data to the NYU Law website. If there is more information that would be suitable and helpful for us to provide, we are happy to consider doing that. For example, we are now looking into posting data of the type found in Table 12 of the NALP form (Source of Job by Employer Type), since that would likely be of interest to prospective and current students. Please let me know if there is additional information that you think would be helpful for us to publish.

Our letter was quite clear about what would be helpful: NYU needs to publish its NALP report. It was a short letter and referenced the NALP report five times. (Read it here.) There is no other way to read any of our three requests and it’s insulting that Dean Revesz would feign ignorance while referencing the very report we requested.

We responded to Dean Revesz, repeating that we believe the entire NALP report should be published and noting what information NYU has not released (along with the sections of the NALP report that contain this information):

• Full-time and part-time employment by required/preferred credentials (“FT/PT Jobs”)
• Classification of business jobs (“Business Jobs”)
• Position held in a law firm (“Type of Law Firm Job”)
• When the job was obtained (“Timing of Job Offer”)
• Salaries by job type (“Employment Status Known” and “Employment Categories”)
• Salaries by location (“Jobs Taken by Region” and “Location of Jobs”)
• Salaries by firm size (“Size of Firm”)

Dean Revesz’s response to our last communication was disappointing:

As I indicated in my last note to you, we will continue to add data to the area of our website that contains employment statistics.

We must read this to say that NYU has no plans to publish the additional information we have requested, even though it is of value to both the legal profession and law school applicants. The data NYU indicated it was “looking into” publishing 14 months after collection–the source of job by employer type–has not been added to NYU’s website despite more than another month passing. The immense intellectual, technological, and financial resources of NYU have proven unequal to the task of posting an 8×7 table.

(To see the data NYU knowingly withholds, check out our reconstruction of its NALP report in our data clearinghouse. Fields shown in dark gray represent information NYU possess but has not released.)

“Having trouble knowing what to believe? We have a proposal for Paul Campos: come audit our numbers. We’ll show you a list of all NLJ 250 firms to which we sent associates in 2010 and 2011. Pick a reasonably sized sample from that group, and compare them to firm-verifiable data. Then let us, and the world, know what you find.” – NYU’s Rebuttal

Professor Paul Campos called in to question NYU’s biglaw placement rate, citing a discrepancy between the numbers reported by NYU and by the National Law Journal. NYU’s response was clear:

Come at me, bro!

Something about NYU’s response (I’m a 2008 grad) just didn’t sit right with me. It wasn’t the acerbic tone the rebuttal took. It was that Law School Transparency had already requested NYU be more open about its job placement rates. LST actually made that request twice, and we have renewed that request again today.

Like all-but-six law schools, NYU has in its possession, right now, a NALP report with detailed job placement statistics for the class of 2010. This report contains a wealth of information, including the size of firms students went to work at; salary information for a multitude of categories; if the jobs are full time or part time, permanent or temporary; what states they are in; if the job at a law firm is as a lawyer, a clerk, or a paralegal; if students found the jobs through OCI, a job posting, or went back to work at a pre-law school employer. It paints a very detailed picture of what happened to the class of 2010, but it’s a picture that NYU has decided to not let anyone else see.

That’s what makes NYU’s response to Professor Campos so strange.

NYU professes its openness, its honesty, its transparency. With one hand it extends an offer to verify that things are as it says they are, but with the other hand it folds up the numbers and locks them away. “Come audit our books…. But not those books!”

Having trouble knowing what to believe? We have a proposal for New York University: disclose your numbers. You’ll publish the employment data collected by NALP for the class of 2010 (and for 2011 when you receive that report this summer). Then let us, and the world, know what you already know.

Read the letter after the jump

• Letter from LST to NYU

In this month’s National Jurist, the magazine’s editors used the data from our Live Transparency Index to grade how transparent each law school website is. While we do not encourage or approve of ranking or grading schools based on the number of criteria met, because some criteria are more important than others, we did provide National Jurist with a spreadsheet of the data to ease their workload. These are the same data that were available on the Live Index as of the end of February 2012. As such, some schools have updated their websites since that time.

In other news, another school provided LST with its class of 2010 NALP report: St. Mary’s University. Over the next few weeks, we will be encouraging a letter writing campaign to convince other schools to provide us with their class of 2010 NALP reports. In the meantime, we are busy inputing the data from all NALP

We have a few updates on our progress towards greater law school transparency. The first is a rundown of voluntary website improvements made by law schools in advance of the ABA standards reform. The second is to announce that LST has obtained its 40th NALP report from an ABA-approved law school.

Transparency Index: Class of 2010

Back in January, we released the Transparency Index, an index of every ABA-approved law school website. It measures how transparent law schools are on their websites in detailing post-graduation outcomes for the class of 2010 through the lens of 19 criteria.

We said:

Taken together, these and other findings illustrate how law schools have been slow to react to calls for disclosure, with some schools conjuring ways to repackage employment data to maintain their images. Our findings play into a larger dialogue about law schools and their continued secrecy against a backdrop of stories about admissions data fraud, class action lawsuits, and ever-rising education costs. These findings raise a red flag as to whether schools are capable of making needed changes to the current, unsustainable law school model without being compelled to through government oversight or other external forces.

There is good, bad, and disappointing news. The bad news is that secrecy is still the norm, with schools still opting to selectively withhold class of 2010 information to suit their own interests (and to the possible detriment of prospective students). With this year’s admissions cycle well underway, and admitted students closing in on their final decisions, people could really use the critical information schools have at their fingertips. While we can place some responsibility at the feet of those who will knowingly choose to attend schools that are withholding critical information, their poor choices still stem from law schools’ bad acts, especially when it is clear that many prospective students still do not understand the extent of the gaps in information, which law schools have become so adept at hiding.

The good news is that we’ve seen been big improvements to a number of law school websites following the publication of our Winter 2012 Transparency Index Report. Further, these improvements are likely an underestimation: we’ve only updated the Live Index as we’ve come across evidence of improvements or have been directed to updates (usually by the schools themselves). As more and more schools respond positively to criticism, it is also getting easier to identify who the bad actors are.

• 22% of schools do not provide evaluable class of 2010 information, up from 27%.
• 64% of schools indicate how many graduates actually responded to their survey, up from 49%. Response rates provide applicants with a way to gauge the usefulness of survey results, a sort of back-of-the-envelope margin of error. Without the rate, schools can advertise employment rates north of 95% without explaining that the true employment rate is unknown, and likely lower.
• 39% of law schools indicate how many graduates worked in legal jobs, up from 26%. 20% provide the full-time legal rate, up from 11%. We are aware of no additional schools providing the full-time, long-term employment rate. (It is still just two schools, or 1%.)
• 28% of schools indicate how many graduates were employed in full-time vs. part-time jobs, up from 17%.
• 16% indicate how many were employed in long-term vs. short-term jobs, up from 10%.
• 16% of schools report how many graduates were employed in school-funded jobs, up from 10%.
• 59% of schools provide at least some salary information, up from 49%. But the majority of the 59% of schools (63%) provide the information in ways that mislead the reader, down from 78%.

These are substantial changes. The schools who have made them (list available here) deserve some praise. However, it needs to be restated that every school could meet all 19 of the criteria we used in the Transparency Index, so our praise comes with that caveat.

The disappointing news is that one of the most transparent schools from our original report has decided to be less transparent. The University of Houston Law Center previously met thirteen criteria; it now meets only seven. (Original Disclosure; New Disclosure.)

In particular, Houston no longer shares the percentage of graduates in full-time and part-time jobs, in school-funded jobs, and in full-time legal jobs. It also no longer indicates when the graduates obtained the job (timing), nor how the graduate obtained the job (source). The school now also provides misleading salary information because the school no longer indicates the response rate for each salary figure provided.

When asked why they took such a huge step backwards, the dean of career services cited that Houston was now just doing what other schools were doing. She also claimed it was an improvement overall because it also included 2009 and 2008 employment data, although it is barely more than what’s already available in our data clearinghouse (2008 & 2009).

For the unacquainted, Houston copied the University of Chicago’s presentation standard, and in doing so actually decreased its level of disclosure. We criticized Chicago’s standard back in January for this particular reason:

Last month, the University of Chicago Law School received widespread acclaim for its decision to provide improved consumer information about the class of 2010. We believe Chicago received acclaim because the job outcomes for Chicago’s 2010 graduates appear to be strong relative to other law schools’ outcomes. The positive responses confused the unexpected quality of outcomes, which for Chicago graduates remained strong despite the retraction in attorney hiring, with actual quality of disclosure. Chicago coupled tabular data with language about the need for transparency, leading people to claim that Chicago has set the market. But if every law school disclosed employment data according to Chicago’s incomplete methodology, most would still continue to mislead prospective law students.

The Chicago methodology does not distinguish between full- and part-time jobs, between long- and short-term jobs, or whether positions are funded by the law school. Nor does it indicate the numerator for the salary figures. For Chicago, this is a relatively minor oversight because it collected salary data from 94% of employed graduates. But the further the response rate moves away from 100%, the more important it is that the rate be disclosed for every category that a school provides salary information for. Few, if any, other schools have a response rate above 90%.

Predictably, Chicago’s puffery about its dedication to transparency has done more harm than good.

LST Obtains Its 40th NALP Report

There remains reason to be optimistic, however. LST has obtained NALP reports from six more law schools since our original announcement.

• Charlotte School of Law
• Loyola University Chicago
• Michigan State University
• Seattle University
• Temple University
• University of Idaho

This brings the total to 40 law schools. While such incremental improvements to transparency suggest a long road ahead, we do consider 40 a significant threshold. LST will continue advocating for law schools to share their class of 2010 reports and, when the time comes, officially request the class of 2011 NALP reports too. Accepted students who don’t want to wait until then can start contacting the schools to request the 2011 data, now that we have passed the February 15 reporting deadline. If you are successful in leveraging your acceptances to procure the data please consider sharing it with us and directly with other applicants on popular discussion board sites like TLS.

Update: We were alerted that Indiana–Bloomington had made its NALP report available online prior to this story’s publication. We’ve made this correction throughout this story.

Update 3: To see more schools that have since complied, see here.

On December 14, 2011, we wrote all ABA-approved law school deans to request the class of 2010 NALP report that each school received in June 2011. We are pleased to announce that we have obtained 32 34 of these reports (17.3% of ABA-approved law schools and 17.8% of schools with NALP reports). 29 of the 32 34 law schools sent a report to LST. The other three four were pulled from school websites.

We asked for these reports to help prospective law students find the law schools that best meet their career objectives. Together, these reports provide prospectives access to timely, thorough, and comparable employment information. They make LST’s website an even more valuable source of information for prospective law students. We hope to soon add the employment data contained in the NALP reports to our data clearinghouse. In the meantime, these reports are available here. In addition, the list below links to each school’s NALP report.

Schools Providing the Class of 2010 NALP Report to LST (30):

Arizona

Capital University

Cleveland State

Dayton

Drexel

Florida State

Florida

George Mason

Louisville

Marquette

Memphis

Montana

UNLV

North Carolina

North Dakota

Northern Kentucky University

Ohio State

Oklahoma City University

Oregon

Ole Miss

Penn State

Seton Hall

South Carolina

St. John’s

Tennessee

Tulsa

Utah

Wayne State

William & Mary

Wisconsin

Non-Responding Schools with accessible NALP Reports (4):

Akron

Northern Illinois University

Thomas Jefferson

Indiana — Bloomington

We want to make a special note that six law schools do not submit employment data to NALP and therefore do not have these forms: Notre Dame, Pepperdine, St. Louis, and the three law schools in Puerto Rico. This does not mean that these schools do not have ample employment data, however. Rather, these are the only schools who do not have a prepared form in front of them that can quite easily be disclosed to prospective law students.

Interestingly, the vast majority of schools providing LST with the 2010 NALP reports were public institutions. This may be because these schools interpreted our request as an open records request. Two schools (Wisconsin and UNC) explicitly treated our request in this way.

We received a handful of responses expressly declining to provide LST with the NALP report. Consistent with past communications with law schools, a number of schools indicated that they had meaningful employment information already available on their websites. In almost every case, this was (and remains) false.

In addition, there were two “no” responses that stood out. The full text of the responses is below.

First, Ave Maria argued that its NALP report does not provide meaningful information. Dean Milhizer claimed, “Our school is small with a unique mission, and our employment outcomes are reflective of this.” He suggested that additional information would be needed to assist prospective students. He did not, however, suggest what kind of information would be useful. The school’s website contains little information on what graduates found for work, with much of that information serving to mislead applicants.

Second, Chapman University argued that its report was confidential. Dean Campbell’s response is very misleading. It is true that NALP promises that it will not share the graduate-level data or the reports generated from those data with anyone except the law school. But schools are under no such obligation, either contractually or (for the NALP reports) legally. If it was required to keep the data confidential, Chapman could not provide these employment statistics on its website. Instead, the NALP reports only remain confidential because law schools decline to share them with those who would find the information most useful. Fortunately for prospective students, at least 32 33 schools disagree that the NALP reports are confidential.

Overall, as the Live Transparency Index shows, schools are increasingly sharing more employment information. But the vast majority of law schools still leave critical gaps in their presentation of employment information – gaps which the NALP reports would fill. These 32 34 law schools have demonstrated leadership that is sorely lacking at other law schools. While these schools still need to choose how to share employment information on their websites, they understand the importance of providing free access to comparable information now. Our hope is that these schools pave the way for changes at other schools, many of whom are still acting as if their applicants do not deserve access to comparable consumer information.

Ave Maria School of Law:

Dear Mr. McEntee and Mr. Lynch:

I applaud your efforts to assist prospective law students in obtaining reliable information about the law schools of interest to them. However, Ave Maria School of Law does not believe that its Class of 2010 Summary Report from NALP will provide meaningful information about our school to prospective students. Our school is small with a unique mission, and our employment outcomes are reflective of this. In our judgment, the Summary Report does not provide sufficient information about the types of positions obtained by our 2010 graduates, and so to release the report in a vacuum without additional information would not be of assistance to prospective students.

This year, as in years past, AMSL will comply with NALP’s guidelines on reporting employment outcomes for the Class of 2011, and we will be participating in the ABA’s new annual survey of these outcomes.

Sincerely,

Eugene R. Milhizer
President and Dean

Chapman University:

Dear Law School Transparency,

You have requested that our School of Law send you the otherwise confidential report that we supply to NALP. We have years of experience with NALP, and on that basis, we know that the information we send will be appropriately treated, consistent with good ethics and all applicable federal and state laws. Regrettably, we do not have that kind of basis with your organization. Accordingly, we think it is most appropriate to continue to keep our submission to NALP confidential.

Nevertheless, our School of Law maintains a very informative website, and we post a great deal of data on our graduates’ employment there. That information is available to the public, and we invite you to consult this source if you would like.

Sincerely,

Tom Campbell
Dean