Enter Prospectives

The first prospective student is scheduled to arrive in just a few hours. Let the partying commence, or something like that.

It's always hard to figure out how to interact with prospectives. We do interviews. Out of the 12 visitors, somewhere between a third and a half will actually return in the fall (based on recent entering class sizes of 3 to 6). My input may or may not be useful (i.e., used) when the faculty decide who to accept. Based on recent ruminations about my own acceptance decisions, my biggest role is in helping the prospectives figure out what they will decide if they do get accepted. I can either scare them away or share the misery.

Not that I intend to frighten them off, or am that miserable in graduate school. I intend to be honest, should my opinion be solicited. Not long ago, someone asked if I would still come to this school if I were making the decision now. There is no straight answer for this question. Things have obviously not been perfect. There have been times where the alternative - a professor in the middle of switching schools, where the first year would have been taken up by lab building - seems preferable. But then, I could never really know what that advising relationship or research would have been like. I don't like being so far away from my family, but I prefer the city I live in to the closer-but-blah alternative. I don't regret the decision I made, and am in general quite happy with the way things turned out.

On the other hand, I have a lot of pointers to prospective students. I've heard tell of one particular advisor who does not meet regularly with his students. One was okay with this; one hated it and switched advisors; one asked for regular meetings. It's things like this, specific to the advisor to the department, that would have been useful to know a few years ago. It's always good to have some idea about what you're getting into.

Reviewing: The "Sandwich" Methodology

I am a member of the main organization for my field. Not because I am particularly interested in the organization, or the journals I receive, or because it looks good on a resume. No, I pay $60+ dollars each year for the privilege of being a student reviewer, and spending a few days of my time reading submissions and providing detailed feedback. In the Fall, I review submissions for student grants; in the Spring, I review submissions for several awards or presentation slots in the annual organization conference.

Why pay to volunteer my services as a reviewer? Partly because it's a change of pace from my normal graduate student responsibilities, partly because I'm passionate about making an undergraduate degree meaningful and this lets me train some students on writing and science even when I'm not a TA, and partly because it is interesting. None of the papers I get to review are related to my Area, and it's nice to spend time seeing a difference side of the Field.

Which is not to say that I'm thrilled with some of the requirements of reviewing. The review process is fairly well done. I return a scoring sheet with my final yes/no recommendation, and a one-page review with "a brief paragraph summarizing the proposal in your own words and a couple of paragraphs of feedback". The poor saps who got stuck coordinating each thing review our reviews (with the threat of sending them back for revision), and want them to have both positive and negative feedback. Sometimes this is hard. More to the point, I can't let go of training on cultural differences that says this is confusing.

As an undergraduate, I minored in American Sign Language. Our teachers were all Deaf (capitalization denoting a cultural group rather than simple physical difference), and courses including explanations of cultural differences. Among other things, Deaf people do not see it as rude to ask for information like how much money you spent on something (sharing information being very important in small communities) and do not like the "sandwich" method of feedback that is so popular in business these days. Tell someone good-bad-good, and the result is complete confusion on "am I in trouble? did you like what I did or didn't you?".

The review process wants us to say good things as well as bad, all in one page. Constructive criticism I can do: here's something I had a problem with, here's what I would have preferred to see. Trying to put lots of ego-boosting statements about things that seem so obviously okay/well done is just frustrating. How am I supposed to squeeze details about my substantive comments into a single page if I need to come up with something positive about the methods section immediately before saying "it wasn't there"?

Prospectives Parties

Let the weekend of obligatory party attendance (and free food!) commence.

Thursday evening is the graduate student social, when the prospectives have a change to unwind before some early-morning interviews. This has become an unofficial part of Prospectives Weeding. Last year, I heard (over and over and over), one of the prospectives ignored repeated warnings of "alcohol will affect you more strongly here", got drunk, got sick, and was found naked on the bathroom floor of his host's home in the middle of the night. The graduate students involved begged their advisor not to admit him.

Friday afternoon is a boxed lunch, to give us the opportunity to meet with prospectives in their interview attire instead of fresh off the plane. More to the point, it's free food. I entertained prospectives in my own interview year by my haphazard attempts to identify the ingredients of the fruit cup (turns out those green squishy balls have their own name).

Friday evening used to be a dinner with the grads, followed by a general Area party as a faculty member's house afterwards. Last year, at the request of faculty wanting more time to spend with prospectives, it was changed to dinner with the faculty and the grads were invited to a Dessert Party instead. I think the addition of the dessert buffet was intended to entice the graduate students into attending (free food!).

Saturday is generally not free, but will include group activities like brunch and probably hiking (if the weather is nice - there was snow this morning). Generally speaking, an excuse for not doing research over the weekend.

Why say they're obligatory when there's all this free food enticement? Well, there is a general expectation that current grads will show up for these things since food is provided, although no one is going to be walking around with an attendance sheet. Most importantly, my advisor is interviewing 2 prospectives who would be interested in working with her. Unless I take a fatalistic perspective that my input counts for nothing, I need to evaluate these perspectives to see if there is anything outstanding or insane about them.

So I'll be spending the weekend being social, regardless of how much I would rather spend my time reviewing papers or analyzing data. Truly, it is a great sacrifice.

Graduate School Attrition

For graduate student purposes, The Department is not so important as The Area. Our Department is huge, so huge that it is divided into Areas that have little to no interaction with each other; I couldn't even tell you how many there are, let alone their names. So, even though therea re ~4,000 undergraduates in our Department, it only feels like there are 12-20 graduate students.

enter 2004: 5 start, 4 now (drop out post-master's)
enter 2005: 3 start, 2 now (drop out 2nd year)
enter 2006: 5 start, 4 now (drop out 1st year)

The trend of dropping out earlier and earlier is probably a result of small sample size (I only know about the years immediately around mine), and has to reverse since most of the "enter 2007" class seems to be here (I can't remember if there are more than 4 now, let alone at the beginning of the school year). In the big picture, our attrition rate probably doesn't seem too bad.There are always going to be those who decide grad school isn't for them, or just get sick of it after so many years.

From the very immediate perspective of being one of those who entered in 2005, however, both class size and attrition have tremendous psychological impact. Especially since the two of us who remain have the same advisor. This was great in the "Have you scheduled your defense yet? Oh good, because I haven't either..." sense, but bad in the "Did you know we had to have this paperwork in by today??? Our advisor didn't mention it!" sense.

Now, our paths diverge. The first post-master's step is to propose our comps paper by April (3rd year), and defend it by December (3rd year). But the other student is doing a double major, and gets a one-year extension on the defense. We propose at the same time, but he with the sense of "oh, I've got plenty of time to work on the actual paper". What little support network I have in navigating the deadlines and/or milestones of progress to a PhD will be gone.

On the bright side, most (all?) of the students in the year above me are also double-majoring, and will be defending their comps at my deadline. One of these is also in my lab, so I have a near-neighbor to commiserate with. As long as I can overcome that sense that they had two years to put together these papers, and I only had one, so can I possibly do as good a job? On the whole, I would have preferred connections to the larger Department, or at least more students in my year.

Preparing Future Faculty

According to the Carnegie Classification System for identifying colleges, my university is a "Research University - Very High", corresponding to the old R1 ranking. This means that the university awards academic doctorates, and both gets and spends a lot of money on research each year. There are 96 RU-VH universities in the United States. It's actually a partial sacrifice of anonymity to admit to this classification.

From a graduate student perspective, this is an amazing opportunity. I am learning how to do research at a university (and department) that are dedicated to research. My advisor opts out of most teaching requirements ("buys out" of them in some weird process by having X many research grant dollars), as does most of the department (except for the Instructors, of course). The first year of graduate study was as concerned with completing a research project as it was with course grades, and for the third year and beyond it's only concerned with research (I have to take one course a year - not a semester, a year). I am being as intensively trained in research as is possible. Huzzah.

From a future faculty perspective, however, this is a problematic position. The Carnegie site says that it changed classifications from the familiar R1 because there were almost 4,400 universities to classify. So only about 2% of the higher education institutions RU-VH. Narrow it down to a specific department, and it is a virtual guarantee that no PhD fresh out of grad school is going to get a job at a RU-VH. Which means that I am training in an environment that may very well be nothing at all like the one I eventually end up working in.

These perspectives combine to mean that I am being very well trained on research science, but not so much on the general job requirements with which I will eventually be faced. My perspective on professors as occasional-if-ever teachers is not going to match what I will need to do if I go the academic route. This would substantially change the dynamics of any department.

Fortunately, the Graduate Teacher Program here has a Preparing Future Faculty network (and certificate, should I choose to go that route). There are guest presentations from people at other classifications, trips, even possible mentorship with local colleges of different classifications. I think I can safely say that I wouldn't want to work at an RU-VH even if I could - far too much pressure and competition - so I'm using as much of these resources as I can to try and figure out what kind of academic job I would want to get.

How to Train Undergrads

There are days where I laugh at the idea of a bachelor's degree meaning anything. Usually days where I've been heavily exposed to undergrads.

My advisor has long had an open policy of inviting undergraduate students to conduct honors theses in our lab. My first year here, there were no less than five seniors doing so. After that, with the sudden proliferation of graduate students in her lab (four of us), mentoring undergrads through a senior thesis became part of the undergraduate student training. Not that my advisor forces us to do it; some of us volunteered when s/he became too busy to manage additional students, and now it's become The Way.

The problem is that none of us are exactly eager to mentor undergrads at this point. There have been a variety of...failed or near-failed mentoring experiences. There was the project where we ended up kicking the student out of the lab (or, in lab coordinator terms, "we are going to ask you to leave the lab, yes") for complete dereliction of duty, as our Worst Case Scenario. I am supposed to be helping an undergrad through her thesis defense right now, as a matter of fact, but after a semester of showing up to meetings she wouldn't show up to, holding her hand through such simple things as calculating means in Excel, and asking for progress reports that never got sent, we sat down and asked her if she was really dedicated to this "thesis" thing. She wasn't, and left the lab. There isn't really a problem with this, I just would have preferred to find out earlier. A lot earlier.

On the other hand, mentoring can be very rewarding. Last year I mentored a high school student, who went on to win the local science fair with her project and advanced to the state level. She came here for college, and is now a regular member of the lab. It's not quite a gold standard - it was a poster presentation, not a written thesis and verbal defense - but it helps me remember why I agreed to mentor in the first place.

There isn't really a good way to filter undergrads when they ask to do a thesis. Our lab has an application process, including references and resume and two interviews. We do manage to avoid the most ridiculously incompetent undergrads, but recently we have had a remarkable lack of success figuring out which undergrads are here to bolster resumes or graduate with honors rather than actually being committed to whatever project they undertake.

This year there are four juniors who want to take up a senior thesis project for next year. Four undergrads, four grads. No pressure; it's just that if any of us can't or don't want to take on mentoring responsibilities for next year, one of our dedicated RAs will be SOL. With this pressure, we have collectively revamped the process by which undergrads are accepted into an honors thesis and matched to a graduate mentor.

The key idea we hope to promote is independence, or at least self-sufficiency. Mentoring undergrads who turn to you when they can't figure out how to copy and paste data is more work than we have time for. We also want dedication: students who know how much work a thesis will take and aren't going to quit because they would rather spend their time at choir practice. And, of course, we want to train them on the skill sets they will need, which are supposed to be what makes the thesis worth the laudes.

So we're developing a pre-thesis course, required for thesis hopefuls and optional for anyone else in the lab who might find it worthwhile. There are two parts: statistics and writing. The statistics workshop will introduce students to our stats package (a needed refresher for later analysis) and also test independence with an assigned analysis project to see if students can turn training into knowledge with a minimal of fuss. The writing workshop will provide a basic measure of writing and analytical skills (reviewing a journal article, probably). By these workshops combined, we will test dedication. If you sit through all these requirements, you must really want to do a thesis despite pressures on your time.

Whether this actually weeds out any thesis hopefuls, or we will be faced with the pressure of not wanting to deny thesis chances to anyone, remains to be seen. At the least, I'll be less apprehensive about my presumptive role as mentor for having some idea of what the undergrads can and will do.

Mediated Effects

The really big event of the day - the reason attending the PhD Procrastination talk seemed like such a good idea - was a mediated meeting with my advisor. There had been a few incidents last semester of miscommunication, stress, and extreme uncertainty, which we hoped to resolve by having a third, impartial party direct the conversation. We can judge this to be a success, because there was nothing awkward about walking back to our building together.

The meeting focused on a few key issues, primarily trying to allow for better communication in the future. The first was what exactly my job description is. My advisor went to a school where pay was never linked to whether you were teaching or on a specific person's grant, and didn't realize that juggling perceived research and teaching responsibilities could become incredibly difficult when the advisor judges your progress but the department signs off on your paychecks. So my advisor thinks of us as professionals in training, not as his/her employees when we're officially RAs. I also discovered the role of department evaluations (We get letters once a year saying we're doing a good job. No one knows where these letters really come from, or what they're based on) in adding to pressure and uncertainty.

I had a few meta-cognitive issues with the mediation process. I watched Mercury Rising over the weekend, including the behind-the-scenes features where a psychologist discussed how difficult it was for a 9-year-old actor to stop nodding his head or making any visible recognition that someone was talking to him. Most of us do this automatically; autistic children don't. I therefore spent an entire hour noticing every time I was nodding or adjusting my facial expression in response to something anybody said. There were also a few pop-business-psychology buzzwords that had me wincing internally.

As an interesting tidbit, I learned that way back when, PhD comics saved my advisor's sister's relationship with their parents: the sister when to the same school as Jorge Cham, and took 11 years to graduate; she provided the PhD books to the parents, who accepted that if there were books about how horrible and long this process was, the length of time could be excused. I shall have to remember to keep providing relevant links to my relatives as we get nearer my "probable" graduation date of 2010.

Procrastination Lite

Jorge Cham, the mind behind Piled Higher and Deeper Comics, is giving a talk in my area this evening. This will be his second appearance since I arrived on campus. Fortunately, the first was within months of my enrollment in graduate school, and this one occurs after I've received my master's diploma, so a second visit doesn't make me feel like I've been in graduate school far too long.

The talk itself hasn't changed much. The title ("The Power of Procrastination") is still the same. But then, I'm not going for the talk. I'm going because I've had a complicated and stressful week, and probably need some refresher instructions on how to procrastinate purposefully. I'm going because it's fun to see how the actual talk matches up to the "Tales from the Road" he creates. I'm going to see if the talk has improved over the two years (last time, he had to borrow a computer, and PowerPoint version discrepancies caused his "What I Learned In Graduate School" fancily animated point about learning PowerPoint to fail completely, to tremendous laughter). I'm going to get an autographed copy of the 3rd book, to match my autographed copies of the 1st and 2nd books.

And I'm going because my name has an uncommon spelling, and I want to see if it will stand out in the mind of a man who must sign hundreds of books a year. There's always the possibility of being notable enough to get mentioned in the Tales from the Road myself.

A "Thorny Ethical Question"!?

After discovering a stupid mistake in what's supposed to be a trivial matter of my Me,Y & Z paper in preparation, which fortunately didn't result in any qualitative changes in my results (things that had been significant/meaningful were still significant meaningful), we discovered that the same error was in Y & Z (2007). This time the re-analysis was bad; what had been reported as significant was in fact not quite significant. The trend was there, my results showed the effect was real, but the published data set would normally have been presented as "not meeting conventional levels of significance" and so wouldn't have gotten published.

Y, my co-author and lead author on the first paper, had the following response:

"What do you think? Are we obligated to point out the PS flaw in the earlier paper? Would doing so help or hurt the current paper? A thorny ethical question..."

And I went off and had a mild conniption fit.

I was fully on-board with Y's attempts to find a spin to put on the sudden non-significance of the results. The finding is valid, check this other paper; there was this or that confound so we can explain why those results didn't actually turn out. But the suggestion of just ignoring the problem, even on the basis of "more recent/better studies do show the effect is real..." entirely set me off.

The problem is that the original paper, and our summary of the original paper, make a great deal of "is significant even controlling for PS". And it turns out that there was no control of PS at all, there was just control for some nonsensical number value they/we thought was PS. The stats show the important effect is still significant when PS isn't controlled, but all those arguments about how it's beyond and better than this trivial measure are suddenly invalid.

Z, the established name on both papers and my advisor, has not yet weighed in on this issue. The two dozen emails we sent on this matter had panic only in the body, not the subject lines, so s/he prioritized other things and may not even know about the problem yet. I'm hoping, almost desperately, that Z will state that they must issue a correction about the recently-published paper and address this in the current manuscript. I don't care if our almost-ready-for-submission paper has to be changed drastically over another three months of drafts in some attempt to salvage the original set of results. I am going to have a mild nervous breakdown if my advisor decides to let wrong results stand. It's bad enough that the possibility of errors in the papers I read looms large in my mind every time I even glance at a journal. I will lose all my faith in science if faced with respected members of the field explicitly deciding not to correct mistakes they find after publication.

The arguments against correction would boil down to this: that original data set isn't significant, but we can look at Me, Y & Z to show that the effect is real, and the original data would have been significant if we'd kept the experiment going a little longer. To me, this is an excellent basis for the letter of correct, just lacking the specific details, but is completely unsatisfactory for not sending a correction at all. It's the slippery slope of ethics. If people don't correct mistakes that change the meaning of the results for a "good" reason, they might decide not to correct results for a "bad" reasons. Full disclosure is one of the foundations of science, that allows us to decide whether to trust the results (of a specific paper or an author's body of work) and figure out for ourselves what might really be going on instead of taking the author's word on it.

Now, if my advisor argues against making a formal correction, do I go to someone about it? That's where the nervous breakdown comes in...

Data Panic Attack

Lab meetings are very useful things. Yesterday we presented our paper (X, Y, & Z, in preparation) to the lab, my fellow graduate students and an assortment of undergraduate RAs. And one of the graduate students asked a very simple question that has snowballed into massive implications for the published paper (Y & Z, 2007). Meaning, the published data may turn out to be wrong. What was reported and published as significant, and thus meaningful, may turn out to be beyond the accepted cutoff for significant, and thus not meaningful at all.

What exactly is wrong? It was really quite basic. To answer the key question in both of these papers, something has to be factored out. I got quite sick of writing "even controlling for [average z-score]", but it was really an important point. The grad student asked a very simple question about what the difference between groups on this z-score was. And we discovered that the z-score measure was wrong.

For those not in the know, a z-score makes data relative. It tells us how far from the average a certain score is. If the z-score is -2, it's very far smaller than average; if it's 0, it's exactly average; if it's +1.5, it's higher than average. The problem is that I told my statistics package, "give me the z-score for Measure1. give me the z-score for Measure2. give me the average of those z-scores", not realizing that "smaller than average" is good for Measure1 but bad for Measure2. So this average z-score is 0 if both scores were good, or if both scores were bad. In sum, were weren't "controlling for [average z-score]" at all, we were controlling for something entirely nonsensical.

The good news it that this doesn't affect my data. I fixed the z-scores. The statistics actually turn out slightly better for all the critical measures, so I just had to update a lot of post-decimal point numbers to reflect the changes.

The bad news is that, as far as I can tell, this really does affect the original, published study. The statistics have gone from being "significant" to being what we grad students usually term "marginally significant" when forced to present something, anything, to the rest of the department. It's enough to make us think the effect is real, but nowhere near strong enough for publication.

I could be wrong. I hope I'm wrong. I am waiting confirmation from Y that I understood his near-incomprehensible column headings on the original data file. With any luck, I pulled the wrong column for the raw data, and the right column will magically leave the important effect as significant. Maybe I just have too much of an ego to doubt my interpretation of the column names, but I don't think I'm that lucky.

This is too much for a graduate student working on a first publication. There aren't even any poster presentations on my CV, and now I may be about to find out what happens when you find a massive error in data after something has already been published. This strikes me as being rather backward in training. I'd prefer to have the confidence of an actual publication before discovering that publications can be just plain wrong and perhaps shouldn't be trusted.

At least I can console myself that the "obvious" error in the z-score calculation was also made by Y, who was an experienced post-doc at the time (now an assistant professor). I'm sure this will be great comfort as I spend the next 24 hours stepping through each data calculation and analysis for the nth time just to make sure that all the statistics are as accurate as we can make them.

Presidential Debates

The appointment of the University President is, locally and at the moment, garnering more attention than the campaigns of the US President hopefuls. The local news reports that students are staging demonstrations, faculty have considered formal resolutions upbraiding the "search" committee for providing only one finalist, and the College Republicans have their own letter-writing campaign (in support of the nominee).

The students mainly object to a candidate who has a history of right-wing conservative politics, including a previous gubernatorial campaign. The faculty object to "appointing administrators who couldn't even qualify for tenure", since the candidate has only a Bachelor's degree. The College Republicans argue that he has a history of successful fundraising, and that's all that should matter. What else is a president for, anyway?

I agree with earlier protests that a "search" committee should be capable of finding more than one finalist for a position; having only one makes me wonder how many people would actually be willing to associate themselves with our university. I'm less swayed by the argument that an administrator should qualify for tenure. I may be the only member of my family with a master's degree, but I don't think any less of the capabilities or intelligence of my family with only bachelor's or "some college" education. The only job I should be more qualified for, since my master's isn't in a business or engineering field, is professor. Not president or fundraiser.

That said, I'm not going to the meet-the-candidate forum this evening, and wouldn't even if I didn't have prior plans. The Board of Regents may or may not be swayed by student and faculty concerns; I'm inclined to believe "not". Aside from a continuing cynical attitude toward bureaucracy in general and academic politics in particular, I still don't find University President worth protesting on as little information as students have been given. If we were in a fight as symbolic as Deaf President Now, I would join in, but there is nothing symbolic or meaningful in a right-wing politician being appointed university president in a conservative state during a national battle between liberals and conservatives.

Circle of References

My thesis is being turned into two submissions. They cover two separate ideas, two separate sets of analyses, even two mostly-separate subsets of tasks. But just to demonstrate how interconnected the papers are, they refer to each other. It's not as bad as it could be, which would be each referencing the other as being "in preparation". The short conference submission referenced "X, Y, & Z (in preparation)"; XYZ references "X & Z (submitted)". Still, I couldn't help but hear theme music from "The Lion King" started running through my head. If I had any brainpower left over from writing them, I'd try to parody the music into "The Circle of References".

University Politics

In honor of Super Tuesday, let's talk Presidents. Only not national presidents...university presidents.

My university is in the process of getting a new president, and has recently named the lone "finalist" for the position. Somewhere on campus, someone cares enough about the selection to deface the man's portrait with a complaint about the selection. I, however, could not tell you who is currently serving as university president, whether he's a real president or just a stand-in during the search, or even how many university presidents we've had in the three years I've been enrolled in this school.

I have no idea what a university president does. Obviously they're enough of a figurehead to cause an uproar if they start spouting nationally derided statements (thinking the Harvard president's comments on women in science), but from a personal standpoint it feels like department chairs and department policies have much more of an impact on my own life. This may be an entirely erroneous assumption, but it's not as if the university president is single-handedly setting tuition rates or graduate student stipend rates. None of the undergraduates in my lab knew, either, and they're the ones who don't have tuition waivers and have more of a vested interest in university decisions.

Actually, after trying to find out what a university president is supposed to do, I'm still neither impressed nor concerned. It seems that the quality of university presidents is quite poor across the nation, so I shouldn't suffer any lack of extra prestige for my degree because it came from a specific university. Besides, the quality of research and my advisor's stature are going to be more important in any academic career, and the degree itself should be all that matters for a business career.

Still, I'd respect the opinions of people who object to our de facto new president if they were slightly more elegant or at least less criminal in the expression of those ideas. I'd put more thought into the governing of my school - I've already been completely unmoved by signs attempting to instigate the student body into taking action against tuition increases - but I can't quite convince myself it's worth the effort. Perhaps when I feel the impact of decisions made outside my department (the possible eventual professor job), it will seem important, but for now it's just some figurehead name.