Working the Conference Ecosystem – More on the Review Process
I have submitted to some conferences for 3 years in a row without any success. It sucks. I figure I haven’t yet cracked the code for what they are looking for. That brings me to the subject of the conference review process. Here are some of the processes I’ve seen (and I’m sure there are more):
The proposal “black hole” – you toss in your submission, you get only one shot and get no feedback or questions. 6 weeks later you receive an accept or reject email.
The proposal “incubation” process – usually a high feedback process with a supporting submission system that provides lots of peer review and allows for some evolution of a proposal.
The “invitational” process – just invite the presenters you like. No chumps allowed.
The Black Hole
I think the first system, the “black hole”, is probably the most common. The proposals all are batched up and reviewed by a committee of some sort. As an outsider submitting to this system you have absolutely no insight into the decision making process. You really don’t know how they make their decision. All you know is that after a set period of time you receive an accept or a reject and that is pretty much the end of the story. Not much learning takes place on the part of either party in this system. There really is not much opportunity for learning to take place when there is no feedback. The nice thing is that the anxiety is actually kept to a minimum. I know that sounds crazy, but compared to some other processes, the “black hole” proposal process is relatively painless. You are in or you are out. No convoluted explanations, no bogus feedback by reviewers who don’t know what they are talking about, no agonizing over a million little revisions. You are in or you are out – period, full stop. Sometimes ignorance really is bliss. There, I actually said it.
Incubation
The second system, the “incubation” proposal process, is very high feedback. My personal experience has been that this is a bit of a mixed blessing. Proposals that are promising, but perhaps would not otherwise be considered, have a chance to be improved and matured with a rigorous feedback process. I find this possibility very exciting. I like the idea of taking a proposal, perhaps from someone relatively new, and helping them to develop it into something really great. I think there is a place for a peer review system that provides new talent with guidance and helps them to bring their ideas to a new audience. As potentially pompous as that may sound, I like it! And in the ideal world this is just how it works.
However there is a dark side to these “incubation” proposal systems: sometimes the feedback really does more harm than good. In these sorts of submission systems I think there is a very high bar that the reviewer has to meet. Poor feedback is almost worse than no feedback at all. Often times these systems allow public feedback from the general audience. I have mixed feelings about this sort of feedback. While I think it’s valuable in some regards, some of my worst, most caustic, useless feedback has come from these sorts of systems. People who are just venting their garbage. I guess as a someone who is proposing to a conference using an “incubation” system, you need to have thick skin. You can’t be too sensitive about your feedback. You need to be able to take the feedback and filter the wheat from the chaff. That’s probably true in any system, but certainly more so in one that allows unregulated public feedback.
Furthermore, as I mentioned before, there is a higher expectation for the quality of the feedback you will receive in a conference like this. As a reviewer, it’s a lot more work – a lot more dialog is necessary in order to help someone develop a proposal that needs significant changes in order to be approved. As a reviewer, that’s your job. To keep coming back and providing guidance and critique as the submitter makes changes. I’m always a little amazed when a submitter receives feedback and then doesn’t update their proposal. Feedback, even tough feedback, generally means that the reviewer is willing to continue the dialog. So go for it! Make the changes and then ask for more feedback! That’s what a healthy dialog looks like! Keep pushing until the reviewer gives in! After all, if you don’t respond to the feedback, you’re proposal is very likely dead.
Perhaps the worst case is when there is no dialog at all in the “incubation” systems. It happens. It’s the, “Great proposal, but not for this stage.” kind of feedback that will drive a submitter stark raving mad. This is a flaw in the reviewer, not the submitter. The reviewers need to work this stuff out and be able to give a coherent message to the submitter. Even worse, there have been times when there has been just a couple of, “great idea” comments and then your proposal is rejected. Again, this is a failure of the reviewers – reviewers really owe the submitters more than that.
Now I appreciate the fact that reviewers are human too. Therefore, I don’t expect miracles…often. But like in any herd there is safety in numbers. (Did I really just call a review team a herd?) As long as you can provide multiple reviews it is much more likely that at least one of you will come up with a cogent, intelligent set of critiques or feedback that resonate for the person who submitted the proposal. I’m not the most experienced reviewer, but I feel best when there are upwards of 5 reviews per proposal at minimum. Then I feel like a sufficient number of eyeballs have looked at the proposal and that there is a reasonable chance that the “wisdom of the crowd” will kick in and enable some useful dialog.
Invitation Only
Finally, there is the invitation only system. I really don’t have any experience with this, but I know of conferences that are run this way. I think on the one hand it offers a certain degree of reliability. As a conference organizer you are interested in keeping the quality high for your attendees and you aren’t interested in taking many risks. So, you stick to those you know and their friends and this system does seem to work. The flip side is that you aren’t necessarily going to get a lot of new voices and new ideas. Not every conference values innovation like that, but I suspect that for the conferences that do want to be on the cutting edge, you can’t afford to just invite those you already know. You need to take a few risks.
So Many Conferences, So Little Time…
One other thing that I try to keep in mind when submitting to a given conference is that there are a lot of conferences to choose from. Some are harder to get a submission into than others. A local open space conference is a great place to try out ideas and see if there is traction in the audience for them. The bar to entry is extremely low.
Then there are regional conferences where there is some review, but often they are quite easy to get into. The audience is still reasonably small, and there isn’t the intense competition to be a speaker. These regional conferences offer a great deal and can be a nice middle ground where you can continue to grow and nurture presentation ideas and delivery.
Finally, there are the big national and international conferences that garner a large audience and get lots of attention. There is a lot more competition to get submissions into these conferences. If you are coming up with an idea for the first time at one of these large conferences, you probably shouldn’t be too disappointed if it gets shot down for not being well developed enough. You will be competing against folks who have been developing their material at other venues and have refined things pretty well by this point.
I think the person submitting proposals needs to keep some perspective on the overall conference ecosystem in mind when submitting to a conference. A big national conference may not be the best place to float a new, untested idea for the first time. That’s not to say you can’t do it, but perhaps trying it out in a smaller venue would be well advised.
#submissions #Review #medium #small #proposal #ecosystem #conference #large