Computer Science 690B, Winter '17
Course Diary:
Research Methods in Computer Science II
Copyright 2017 by H.T. Wareham
All rights reserved
Week 1,
Week 2,
Week 3,
Week 4,
Week 5,
Week 6,
Week 7,
Week 8,
Week 9,
Week 10,
Week 11,
Week 12,
Week 13,
(end of diary)
Monday, January 9 (Lecture #1)
- Introduction to CS 690B: Content and Logistics (Slides)
- The Research Process: The Ideal vs. (Bitter) Reality
- The reality of research is that many false
starts may happen at any point in the process;
sometimes, one may even have to backtrack to an
earlier stage in the process and begin again.
- Though one cannot ever eliminate such false starts,
one can with forethought and appropriate effort
minimize their occurrence and make the research
process not only shorter but more enjoyable.
- Starting Research: Selecting and Validating Topics
- Crucial part of both thesis and general research.
- The three necessary properties of a good research topic:
- Finding a topic that has not been done before is a central
part of doing original research, and is crucial to
BSch, MSc, and PhD theses as well as academic research in
general.
- The central role of originality is a large part of
why plagiarism is so despised in scientific research.
- Finding a topic that is important is not so crucial at the
BSch or MSc levels (where one is arguably learning
techniques and how to apply them as well as the manner
in which research is conducted), but it is crucial at
the PhD level as well as in academic research in general.
The latter is so because it is important research which
allows publication in recognized venues and the formulation
of quality funding proposals which are highly likely to be
funded, both of which are important in building an academic
career of your choosing (and not merely one that is available).
- Finding a topic that you can work on productively is arguably
the most critical aspect, because without this, there are
no research results and hence none of the benefits described
above that accrue from them.
- Assessing the originality and importance of a topic is typically
done by performing a thorough literature search and consulting
with experts in the topic area; assessing whether a topic is
one that you can work on productively requires (often brutal)
honesty about one's own abilities.
- Failure to do any of these assessments fully can result in much
wasted time and effort.
- Example: The 2001 SAC Incident (Failure wrt Originality)
- Example: The 2007 ICDT Incident (Failure wrt Importance)
- Example: Things I Didn't Do for My MSc and PhD I:
the combinatorics of reticulate evolutionary trees and the
parameterized complexity of evolutionary tree inference (Failure
wrt Doability)
Wednesday, January 16 (Lecture #2)
- Starting Research: Selecting and Validating Topics (Cont'd)
- Four types of research topics:
- Verification of existing work (Old Bones, Old Graveyards (OBOG)),
e.g., rerunning or redoing a simulation study (the latter
if the original code is missing) to see if the same results
are obtained.
- Given that reproducibility (and hence trustworthiness) is a
core aspect of science, this is sometimes necessary.
- Given the importance of the original work being reproduced
and the existence of at least a partial description of
how this work was done, such a topic can be both
important and doable; however, it is not original and
hence often does not get the respect it deserves (and hence
may lose importance).
- These are low-risk research topics which, given their
frequent though undeserved lack of respect, are relegated
to BSch theses or "housekeeping" aspects of academic
research in general.
- New application of existing methods (Old Bones, New Graveyards
(OBNG)), e.g., applying standard genetic algorithm
design techniques to develop a genetic algorithm for improving
the prioritization of orthodontic hamster surgery cases.
- Such topics are typical of basic ongoing scientific
research (cf. Kuhnian Scientific Revolutions) and hence are
the most common research topics.
- By virtue of using existing methods, such work is doable
and has importance and originality in proportion to
the importance and originality of the application area.
- These are low- to moderate-risk research topics, and hence
typical of BSch and MSc theses; are also useful in
academic research in general when results must be
obtained reliably and quickly, e.g.,
conference / funding proposal deadline.
- Development and verification of new methods (New Bones, Old
Graveyards (NBOG)), e.g., developing either a new
type of genetic algorithm or a totally new type of algorithm
period.
- Depending on the degree of novelty of the new method,
such topics are typical of advanced ongoing scientific
research and minor Kuhnian Scientific Revolutions and hence
are somewhat rare.
- Such topics are by definition original and important in
proportion to the advantages and applicability of the
new methods; however, such work may not be doable or
very difficult to do.
- In order to verify the advantages and applicability
of a new method, it is critical to apply this method
to applications to which existing methods have
already been applied and hence which have results
against which the new method can be compared.
- These are moderate- to high-risk research topics (in
proportion to the novelty of the new methods) and are
hence typical of MSc and PhD theses as well as
academic research in general; are most productively
tackled when there is no hurry to obtain results,
e.g., when you have several years grace to
work on a PhD or after you have tenure.
- Simultaneous development of new methods and new applications
(New Bones, New Graveyards (NBNG)), e.g., atomic and
relativistic physics.
- Such topics are typical of major Kuhnian Scientific
Revolutions based on previously unknown phenomena,
e.g., radioactivity, and hence are very rare.
- Such topics are by definition original and important;
however, they may not be doable or very difficult to do.
- These are very-high-risk research topics, i.e.,
nice work if you can get it.
- Two sources of thesis topics:
- Supervisor-selected
- Risk inversely proportional to the degree to which
the supervisor knows both the topic area (and hence
the topic's originality and importance) and your
ability to work productively on the topic.
- Can be completed quickly and is very good for
learning particular methods; however, may be a tad boring
at times.
- The most typical type of BSch and MSc topic.
- Self-selected
- Much riskier, as you have to evaluate originality,
importance, and doability by yourself.
- Is typically time-consuming and a somewhat more
terrifying experience; however, one learns a lot
more, including how to be an independent researcher.
- The most typical type of PhD topic.
- Two sources of research topics in general:
- Open questions in existing work
- Such questions may be published in the literature or
circulating by word-of-mouth among the members of a
research community; access the former by literature
search / reading and the latter by talking to
colleagues at your home university and at conferences.
- Depending on the reliability of the source, topics
based on such questions are important and original,
and there may already be hints on how they might be
doable.
- Note that the effort involved in phrasing a good
open question, i.e., one that is original,
important, and doable, often makes people
(IMHO rightly) reluctant to share them unless
credit for the effort is acknowledged by
co-authorship or some other means, e.g.,
Fellows and Berger 1995 conversation.
- Combination of elements of existing work
- Such combinations arise from the fortuitous collision
within your mind of disparate ideas that you have
encountered, e.g., why hasn't anyone thought
of cooking mushrooms with a blowtorch before?; as such,
one can increase the probability of such combinations
by creativity-boosting mental exercises as well as
by increasing the number of ideas you encounter by
reading and talking to people.
- While combinations may indeed by original, their
originality and importance as well as the extent to which
they are doable must be very carefully assessed.
- Example: $M_l$ consensus tree algorithms (my BSch dissertation)
[OBNG / supervisor-selected; open questions]
- Example: The computational complexity of evolutionary tree
inference (my MSc project) [OBNG / self-selected; combination]
- Example: The parameterized complexity of phonological processing
(my PhD project) [OBNG / self-selected; combination + open questions]
Monday, January 23 (Lecture #3)
- Starting Research: Selecting and Validating Topics (Cont'd)
- Example: The parameterized complexity of of analogy derivation
under Structure Mapping Theory (van Rooij et al (2008))
[OBNG / collaborator-suggested; combination + open questions]
- Example: The parameterized complexity of computer game
agent verification and design (Watson and Wareham (2015))
[OBNG / collaborator-suggested; open questions]
- Example: The parameterized complexity of environment design for distributed construction
(Wareham and Vardy (in preparation))
[OBNG / collaborator-suggested (sort of); misunderstanding + combination]
- Example: The structural approximability of cognitive processing
(van Rooij and Wareham (2012)) [NBOG / collaborator-suggested;
open questions]
- Though a lot of work, the process of selecting and validating a
research topic can become a seductive activity in itself. One must be
careful to always be mindful and assess why you are doing
what you are doing. Remember: your goal is to get a topic
that is original, important, and doable, and if one possible topic
is not working out, move on to another in a timely manner.
- Example: Things I Didn't Do for My MSc and PhD II:
chaos-theoretic aspects of evolutionary inference.
- A final caveat is in order: A good research topic takes effort to
develop and is valuable in itself -- beware of spreading it around
before you've had a chance to work on it fully, as others may not
respect this effort.
- Example: The CWCB 1996 Incident.
- Example: The 1999 PhD Defense Incident.
- Example: The RECOMB 2001 Incident.
- Example: The 2014 Cog Sci Appendix Query.
Monday, January 30 (Lecture #4)
- Starting Research: Timelines and Proposals
- Graduate Degree Research Timeline
- Above is the ideal; may take longer, but should not take
too much longer (cf. my MSc thesis [4 years]).
- Does not include PhD comprehensive exams.
- The timeline for funded research projects is similar in structure
to that above though the specific durations involved vary with
the project length, e.g., OPC (prep + 2 months evaluation +
1 year project), NSERC Discovery grant (prep + 5 months evaluation +
5 year project).
- Non-funded research projects typically do not have timelines or
proposals; however, it may still be worthwhile to do one or both
of these for your own use.
- A non-funded research project proposal can take several forms, e.g.,
working notes, annotated bibliography, scheduled seminar
(like the departmental Graduate Research Forum!).
- Such proposals serve also in spurring research along,
as (unless you are eventually do something for a conference, journal
special issue, or edited book) there are typically no deadlines.
- The graduate thesis research proposal
- Lays out what you want do, how you're going to do it, and
how long you think it will take you to do it.
- A graduate thesis proposal is not so much a contract for specific work as
a demonstration that you know how to conduct research
and have put in the requisite preparatory work, e.g.,
literature search.
- As such, a graduate thesis research proposal is a
promissory note that you're ready to start
research -- whether it is eventually the research you said
you'd do (and in the manner and time you said you'd do it
in) is another matter.
- A thesis proposal is at its heart a sales pitch whose
purpose is to convince others to buy the idea that
you should be allowed to do research -- hence, whatever
makes the proposal more convincing, e.g.,
easy-to-understand stories, backup of claims by literature
citations, believable timelines, is to the good.
- General structure of proposal (single-spaced pages):
- Title
- Titles should be descriptive and relatively short,
e.g., my BSc, MSc, and PhD thesis titles.
- Save cutesy titles for later publications, e.g.,
my original BSch thesis title.
- Abstract (between 150 and 250 words)
- Motivation [~2 pages]
- Describes why this project is worth doing, i.e.,
what will be done in this project that has not been
done before (originality) and why this is important.
- As part of this, should describe previous and related
work as context for proposed work in order to highlight
the hole in previous work that the proposed research
will fill. Do this in sketchier
form here, with more details in the next section of
the proposal.
- A story provides a natural way of organizing this
section (see the short description of an abstract
above).
- It is particularly important to back up statements
with literature citations in this section. Such
citations not only establish the truth of your
statements (Don't Blame Me -- Blame Them) but
also show that you've put in the requisite
preparatory work (Trust Me --
I Know What I'm Talking About).
- Previous and Related Work [~2-4 pages]
- Describes in detail what has been done before that is either
directly or indirectly related to the proposed project,
cf. Motivation.
- It is tempting to just list and briefly describe references,
not unlike an annotated bibliography. However, all
possible effort should be made to impose more
organization and structure, such as a taxonomy of
existing work (possibly organized chronologically to
show how research has proceeded over time).
- Such an imposed organization makes for an easier
read of the proposal, and should also make it much
easier for you to structure both your motivation
and research question sections.
- A well-done previous work section may also be
a (possibly publishable) research contribution in
its own right, e.g., my MSc thesis.
- Ideally this section should be structured to highlight that
which has not been done previously which you plan to do --
this gives a natural bridge to the following sections in the
proposal. Where possible, consider making this structure
a story or narrative, e.g., "Once upon a time there was a
very important problem. Many people have worked on this problem
so far, but they haven't done important thing X. However,
I have a neat plan to do X -- would you like to hear about it?
If so, please keep reading ..."
- Research Question [~1 page]
- Describes what specific question or set
of questions you will be addressing in your
thesis work.
- A research question is intermediate in rigor between a
formal scientific hypothesis and an everyday question.
- The precise form of the research questions depends
on the research topic; are much easier to define if
the research was inspired by open questions in the
literature.
- Example: The two general research questions
underlying all of my parameterized-complexity-based
research relative to an problem X:
- Can X be solved optimally in polynomial time in general?
- If not, what restrictions do (and do not) make X
solvable in effectively polynomial time?
- Every question should be well-motivated by the
preceding proposal sections and be answerable
relative to the subsequently-described methods
to be employed in the research; as such, these
questions are the central pivot on which all else
in the proposal depends and operates.
Monday, February 6
- University closed by snowstorm; lecture canceled
Monday, February 13 (Lecture #5)
- Starting Research: Timelines and Proposals (Cont'd)
- The graduate thesis research proposal (Cont'd)
- General structure of proposal (single-spaced pages): (Cont'd)
- Methodology [~2-4 pages]
- Describes how you will answer the research questions.
- Methods are typically described in general terms, with
appropriate literature citations for details; that
being said, these descriptions must be specific
enough to allow explanations of how these methods
can be used to answer the posed research questions,
i.e., no ``magic occurs here'' processes.
- Note that this section and the following one are the
ones where you convince others that the proposed
research is doable by you.
- Timeline [~1 page]
- Describes your anticipated course of progress
in the proposed research.
- The emphasis here is on ``anticipated'' --
this is research, not engineering, and
experienced researchers know things seldom go
according to plan.
- Can do sophisticated multi-tier graphics here; however,
the simplest form (an itemized list consisting
of month-specific dates with attached research
activities and/or deliverables) is often sufficient.
- The timeline must be believable -- to ensure this, be
as specific as possible when describing research
activities, e.g., no ``magic occurs here''
processes, and budget for some difficulties when
establishing dates, e.g., The Scotty Principle.
- Bibliography (15-30 references) [~2-3 pages]
- Bibliography should be alphabetic by author name,
and all references should be both complete and
correct in details
and consistent in format, e.g., do not
describe conference papers in two or more ways.
- Compiling a good bibliography is a lot of work,
but it plays an important role in convincing others that
you can be trusted to do research; it can also be
a research contribution in itself, e.g., my
MSc thesis.
- Example: My PhD research proposal [3rd draft]
(PDF)
- Remember that you need not write up the sections in the order in which
they appear in the proposal -- in order to establish originality and
importance, Related Work, Research Questions, and Bibliography should
be done first, followed by Methodology (to establish do-ability). Once you
then know where you'll
end up story-wise, one can then do your Motivation and Timeline.
- Allow time to revise your proposal after the first draft of all sections
are finally written -- at that point, you can adjust story and terminology
to be consistent throughout the proposal.
- The general structure above is also characteristic of funded research
proposals modulo differences in the length of particular sections
and the presence of additional sections, e.g., personal
record in previous funded research and the training of Highly
Qualified Personnel (HQP), budgets, budget justifications.
- Note that in the case of proposals involving funding, the extent to which eventual research must follow
that described in the proposal varies with the funding
agency and grant-type.
- Example: OPC / NSERC equipment / Dutch PhD project
proposals vs. NSERC Discovery Grant proposals.
- Example: My 2010 NSERC Discovery Grant proposal
(PDF)
- Wrt non-funded research projects, it may well be worth doing up
specific sections described above (in particular, previous work,
research questions, and bibliography) to help both focus the research
from the beginning and aid in creating subsequent publications
describing the results of this research (which will require
these materials anyway).
- In all cases, be honest in describing all aspects of your proposal,
particularly the extent to which the proposed research is original
and important and your ability to do it.
- Part of this can be including frank descriptions of potential
difficulties and how you will deal with them.
- Given the temptation and incentives to over-sell research by
being less than honest, honesty can set one apart in a good
way and be reputation-enhancing.
- Example: Reviewer comment on my 2010 NSERC Discovery
Grant proposal.
- Doing (and Surviving) Research
- There is nothing like either the joy of research going well or
the despair of research going badly. Though one can never be quite
sure what research will bring result-wise, one can with care and
forethought simultaneously maximize the former and minimize the
latter.
- There are many techniques for carrying out computer science research
-- these techniques are what one picks up in courses
and personal reading and from talking to your supervisor and other
academics.
- We will focus here not so much on the research process but
on how to survive and enjoy this process.
- General Research Etiquette
- There are some basic common-sense rules that should always be
followed when doing research:
- Be honest in performing research and reporting results.
- Acknowledge the contributions of others to your
research, e.g., do not (excessively self-)plagiarize.
- Where possible and prudent, help others in their research.
- Be courteous and respectful to research support staff
We are human, and lapses will occur (particularly wrt (3) and
(4)); however, be on guard for lapses in (1) and (2) ((1) in
particular), as much of research depends on the trustworthiness
of researchers and such lapses can destroy research careers.
- Though life in general may not be fair, aim to create a
peaceable kingdom wrt your research and the people you deal
with in doing that research.
Monday, February 20
- Midterm break; no lecture
Monday, February 27 (Lecture #6)
- Presenting Research I: The Joy of Peer Review
Wednesday, March 1 (Lecture #7)
- Doing (and Surviving) Research (Cont'd)
- Graduate Thesis Research
- The Thesis Lifestyle
- The Pace of Thesis Research
- Work hard and work consistently, i.e.,
try wherever possible to avoid last-ditch
24/7 work binges.
- Quite aside from keeping your supervisor happy
(by allowing regular demonstrations of research
progress), this is an excellent strategy for
ensuring that you always have additional time
to cope with unanticipated difficulties
- The Necessity of Not Doing Thesis Research
- Make sure (if you have to, by scheduling it)
that you have regular breaks away from your
thesis doing something that you find relaxing and
enjoyable.
- This not only includes daily and weekend breaks
but vacations, e.g., visits home.
- This helps give you both the energy to do good
research and necessary perspective on what you are
doing research-wise.
- Oddly enough, the above applies even when research is
going very well -- the endorphin rush of good research
can be seductive and potentially damaging, e.g.,
death by joy.
- Example: My 2015 Summer (Non-)Vacation
- The Necessity of Setting Personal Priorities wrt Work
- There are plenty of people out there who are more than
happy to tell you how much you should work (and
conversely how little you should be away from work).
- Time spent working has both benefits, e.g.,
faster thesis finishing time, more publications, and
drawbacks, e.g., broken relationships,
personal depression / burnout (see below).
- Example: Cases in point: Dan Brooks,
Tao Jiang, and Bill Threlfall.
- You need to decide what your priorities are and
choose the tradeoffs between work and life that are
right for you before others do it for you.
- During graduate studies, this balance is particularly
hard to strike given the unequal distribution of
power between students and supervisors; this is yet
another reason why it so important to maintain
cordial and open relations with one's supervisor
(see below).
- All of the above can benefit by making sure that you are
good at allocating and managing time both to research
and life in general -- anything that helps in this is
to the good, e.g., training in Time Management /
Meditation / Mindfulness.
- The Care and Feeding of Graduate Research Supervisors
- Ideally, you should have a good intellectual and
personal fit with your supervisor, i.e.,
they should know the things you need to do research
and you should be able to talk to them.
- The former helps your research go forward; the
latter helps when your research has problems.
- Ideally, you should have regular (at least weekly or
bi-weekly) meetings at which you discuss research (and later
thesis) progress.
- Regularly-scheduled meetings are a great way to keep
up the pace of research.
- Try not to dread these meetings -- if there has been
no progress, use the meeting to figure out why this
occurred and what can be done about it.
- As in any relationship, if problems arise between you and
your supervisor, talk them out at the earliest possible
opportunity when at all possible.
- Student-supervisor relations are seldom perfect, and
the causes of deviations from the ideal as well as the
degree to which these deviations are problematic
vary wildly (Tolstoy's Dictum)
- If your supervisor is bureaucratically challenged wrt
regulations and deadlines, make sure that you know
what the relevant regulations and timelines are and
make sure that you follow them.
- If your supervisor is not making time to meet with
you or to give you timely advice and feedback, do
not let them off the hook -- their time is your
right as their student, and patiently but firmly
request that they make time for you.
- If your supervisor is not a good intellectual or
personal fit, get the help you need elsewhere,
e.g., supervisory committee members (PhDs),
other faculty members (departmental and otherwise),
fellow students of the same supervisor, fellow
students, friends and family.
- If there are irreconcilable differences between you and
your advisor or the relationship between you becomes
abusive, talk to your graduate officer immediately
for advice.
- Though there may ultimately be little that can be done
to compel a tenured supervisor to behave, not letting
others know what is going on will only allow things
to get worse; moreover, the assistance of these others
will be necessary if you need to switch supervisors.
- Switching supervisors is a common fantasy in times
of trouble, and it can be done and survived -- however,
it is even in the best situations disruptive and
should only be invoked for real if there is no
other solution.
- Writing Up the Thesis
- Theses take different lengths of time to write --
perhaps the only common feature is that all theses
take longer to write than you think they will.
Hence, start thesis writeup as early as possible
and budget time for problems that can arise.
- Decide on your typesetting system up front.
- The main choices in Computer Science are MS Word and LaTeX,
with research areas varying in their preference.
- Make sure that the conventions you adopt, i.e.,
for tables and figures, citations, and bibliographic
references, are consistent with
those mandated by your university.
- Example MUN Thesis Guidelines (webpage)
- Thesis templates (when available) can be invaluable.
- Example Unofficial MUN Thesis Templates (LaTeX)
- The standard thesis format looks strangely like
that for the thesis proposal (introduction,
background (including previous work), methodology,
results, discussion, conclusions, bibliography).
- This standard format van vary according to research
area, e.g., software engineering
vs. evolutionary algorithms vs. computational
complexity analysis vs. theoretical computer science.
- Your supervisor is typically your best guide to which
thesis format you should use.
- Theses need not be written up in either the final order of
sections or at the final level of detail.
- Consider first doing brief background and methodology
sections and only fill these in with details after
you have written up your results and discussion (at
which point you will know the details you need to
put into the background and methodology).
Conclusions and the introduction can be done last.
- As terminology and story are especially prone to
vary with time over long-duration writeups like theses,
allocate time at the end to make both consistent
throughout.
- Never make the bibliography the last thing you
do -- always add all references in full as you cite
them in the thesis drafts.
- Any time you spend adding full references that are
later deleted is typically vastly less than the
you spend trying to recover references that you
forgot.
- The above is also a good reason why you should
never wait until the text is finalized to add
literature citations.
- Make sure you acknowledge text, tables, and/or figures
in your thesis that are derived or taken from other sources.
- Small pieces of text, e.g., 1-2 sentences,
can be placed in thesis text by enclosing them
in quotation marks and giving a literature
citation with page numbers at the end. Such page
numbers can be done in LaTeX using the optional
square-bracket argument for the \cite command, e.g.,
This approach is not commonly used, in large
part because of the opinion of practitioners
such as Wareham who claim ``... that the whole
thing is nonsense'' \cite[page 23]{War01}.
Larger pieces of text should be offset from the
thesis text, again with a literature citation
with page numbers at the end. Such offsetting can be
done in LaTeX using the quotation or quote
text environments.
- If tables or figures are adapted from other sources,
e.g., the table or figure is simplified
or enhanced, it suffices to acknowledge this in
the caption, e.g.,
\caption{A simplified model of a neuron (adapted
from \cite[Figure 4]{SNW98}).}
If the table or figure in your thesis is exactly the
same as it appears in the source,
e.g., you've re-used the JPEG file from the
source, remove "adapted from" in the caption
acknowledgment above and get written permission
from the author and/or publisher to re-use the
table or figure. Failure to do this can result
in the rejection of your thesis on copyright
grounds.
- Given the severity of the repercussions for not
handling such matters correctly, if you are at all
in doubt about how re-use should be acknowledged
wrt thesis text and obtained permissions, consult
your supervisor and the MUN copyright office
(webpage).
Monday, March 6 (Lecture #8)
- Doing (and Surviving) Research (Cont'd)
- Graduate Thesis Research (Cont'd)
- Writing Up the Thesis (Cont'd)
- Solicit feedback from your supervisor and/or supervisory
committee on individual chapters (and, when they are
critical, individual sections) as they are written,
i.e., avoid Big Bang Writeup.
- This ensures that you do not drift too far from
standard formats and/or supervisor preferences,
and can save a lot of work.
- This also allows early detection and resolution
of format disagreements between co-supervisors
as well as simplifying final thesis checking before
submission for examination (see below).
- Theses at MUN have been rejected because of poor written
English. Hence, if you need help with your written English,
get it.
- Some supervisors provide this; in other cases, use the
Writing Center (webpage)
or hire an editorial consultant.
- The Writing Center is free but may take a while;
editorial consultants are not free but the benefit of
feedback that is speedy (and possibly better-suited
to you and your thesis) may make it worthwhile.
- Some people do this throughout the writeup process while
others only do it for the final draft prior to thesis
submission -- I prefer the former, if only because
Englishification of a large document all at once
can take a lot of time.
- Be aware of your intellectual property rights as regards
your thesis.
- Example: MUN thesis intellectual property rights
(PDF)
- Depending on circumstances, you may not want to put
everything you know or derive in your thesis.
- Example: My 1990 MUN M.Eng. Experience.
- Thesis Submission, Examination, and Defence
- The process of submitting, examining, and defending
theses takes different lengths of time --
perhaps the only common feature is that this process
always takes longer than you think it will.
Hence, you need to start this process as early as possible
and budget time for problems that can arise.
- The Official Timeline (MSc):
- Examiners selected and appointed by SGS Dean (1/2 month)
- Formal submission and examination (1-2 months)
- Notification of results (acceptable, minor revisions,
major revisions, fail)
- Final submission to MUN (6 months for minor
revisions; 12 months for major revisions and
re-examination)
Remember also that a departmental seminar must be
given immediately prior to (2).
- The Official Timeline (PhD):
- Review of thesis by supervisory committee (1-2 months)
- Examiners selected and appointed by SGS Dean (1/2 month)
- Formal submission and examination (1-2 months)
- Notification of results (acceptable, minor revisions,
major revisions, fail)
- Defence and final submission to MUN (oral
defence (+1 month), then 6 months for minor
revisions; 12 months for major revisions and
re-examination)
- The official bottom line is to plan on 1-2 semesters to
complete the process. However, for various reasons, it
can take longer, e.g.,
- Your supervisory committee may be away and/or busy
(the latter during term, the former over holidays
(Xmas, summer))
- Paperwork requiring multiple signatures
among multiple offices takes time (especially if you
are trying to get it done between semesters or during
holidays (late summer, Xmas)).
- Examiners may be busy or unresponsive.
- It may be problematic scheduling a multi-person defence
(though this is much better now with e-defences).
- Your own university paperwork may not be in order,
e.g., outdated program of study, unpaid fees.
Given the above, the best way to avoid such additional delays
is to first make sure that all your paperwork is in order and
then initiate the submission process early in the Fall or
Winter terms.
- The co-operation of supervisors and your supervisory
committee is crucial during this phase
of your thesis; hence, all advice for dealing with
supervisor problems given above goes double here.
- Students of bureaucratically challenged supervisors
need to take particular care and be prepared for
the non-standard, e.g., helping to prepare
your own list of examiners (!).
- Though this is a complex and scary process, don't panic --
if you have put in the requisite work and care, things
will be okay.
- In particular, the PhD defence is often an
anti-climax -- my own experience and that of many
others I've talked to is that afterwards, one's
second reaction (after the initial "Yahoo!") is
"What was I so worried about?"
- Use as much stress as you need to make sure that you
are prepared -- any stress after that is superfluous.
- The Eternal Question: To Teach or Not to Teach?
- This is more an issue for PhD than MSc students;
however, it is something everyone should consider
if they are looking at a career in academia.
- Benefits: Teaching a course not only helps you decide if
you can and want to teach, but it is also an
invaluable enhancement of your CV when you go to
apply for academic jobs.
- Drawbacks: Teaching well (especially for the first
time) is very, very time-consuming, and
can easily (and seriously) delay your thesis
research and writeup.
- Example: Patricia Evans and myself.
- Given the above, whether or not you should teach
during your graduate studies must be discussed
and cleared with your supervisor.
- If you do decide to teach, it would be well worth
your time to build up some expertise by taking a
a course on teaching, e.g., Teaching Skills
Enhancement Program (TSEP) (webpage)
- Dealing with Graduate Research Burnout
- Not to confused with End-of-Thesis Psychosis (ETP),
e.g., OCD + paranoia + manic/depression, which
everyone gets in the high-stress situation that is
the end of a thesis.
- Burnout is much more severe and has a variety of symptoms;
what is most characteristic is a lack of research and/or
thesis progress.
- There are a variety of causes of burnout, e.g.,
bad choices (topic, scheduling), bad luck (research,
accidents, family, relationships)
- Example: My BAh thesis burnout.
- Example: My PhD thesis burnout.
- Tips for averting burnout
- Work hard and consistently to cope with
unanticipated difficulties.
- Maintain good and regular interactions with your
supervisor.
- Schedule and religiously take downtime.
- Keep a progress diary (and read it).
- What to do (and not to do) when burnout happens.
- DO recognize the signs early.
- DO ask for the help you need (and use it).
- Depending on the causes of your burnout, talk to
the appropriate people, be it your supervisor,
friends, or family, or
mental health professionals, e.g.,
- MUN Student Wellness and Counseling Center (webpage): 864-8874 or 631-0500
- Mental Health Crisis Line (webpage): 737-4668
- DO NOT be ashamed.
- DO NOT ignore it.
Monday, March 13 (Lecture #9)
- Doing (and Surviving) Research (Cont'd)
- Non-thesis research.
- You will typically have more choice about what you do
and a longer time in which to do it after you complete
your theses, but depending on your job, you may have much
less time in which to do it than you did during your theses,
e.g., my niece's evaluation of my academic working life
in 1999.
- Good habits wrt managing your time and setting priorities
may be even more crucial after your theses than they were
during your theses.
- This is yet another reason why it is a good idea to develop
good habits wrt such matters during your theses.
- The Eternal Question: To Collaborate or Not to Collaborate?
- After (and to a degree, even during) your theses, you have
the option of collaborating with others in doing research.
One's first experience is most often collaborating with
your supervisor and your fellow supervised grad siblings;
this broadens later to others.
- There are a variety of types of collaboration, varying
for example in the number, diversity of expertises and/or
geographical locations of the people involved and the
duration of the collaboration.
- Larger (especially geographically distributed)
collaborations require much more management and
co-ordination than smaller collaborations.
- Collaborations involving diverse expertises require
initial and ongoing establishment of common ground
that may not be required if all collaborators are in the
same research area.
- Long-term collaborations require a much better
intellectual, work-habit, and personal fit between
you and your collaborators than short-term
collaborations.
- Benefits: Research can go faster and, with the talents
of others, more robustly and in far more directions
then you could have done on your own.
- Drawbacks: Relationships with collaborators must be
carefully co-ordinated and nurtured to result in
productive and enjoyable research; writeup must also
be co-ordinated very carefully, e.g., who
has The Writing Stick for which part and who sets the
overall tone and style of the writeup?
- Like many other types of relationships, collaborations
do not always work and even if they do, they may not
work for long.
- Example: My own collaborative relationships with
Marco Cesati, Gianluca Della Vedova, Patricia Evans,
Yvan Rose, and Iris van Rooij.
- Whatever the form of collaboration, establishing who will
be authors on any produced work and what the author order will be
is very important.
- This is because author order means different things even within
subareas of the same discipline let alone in different
disciplines and these things can have an impact
on co-authors getting tenure and/or grants,
- Example: The 1992 Wood's Hole biologist conundrum.
- Example: Theoretical vs. non-Theoretical Computer Science.
- If you are a graduate student, be aware that there are no
standard guidelines wrt co-authorship with your supervisor;
hence, when you start producing papers, it is worth having
a frank chat with your supervisor about this.
- Example:: Mike Fellows, Ulrike Stege, and Ken Kent.
- There are benefits and drawbacks to every co-authorship policy. and
there will always be people who will tell you what you should
and should not do. It is therefore important for you to
(as with our previous discussion on establishing work / life priorities)
decide early on what is right for you before other people do it for you.
- Though collaboration (particularly between disciplines or
with industrial partners) is very strongly encouraged
these days, you must ultimately decide what is right for
you and the research life you want to lead.
- Unfortunately, burnout is not restricted to theses but can occur
at any time during research; hence, all advice on preventing
and dealing with thesis research burnout given above also
applies here.
- Once one has done research, it is a good idea to write it up for publication.
- This is important because publications are a measure of research success
in academia -- your publications help you get funding and PhD offers when
you are a graduate student and job offers, funding, tenure, and promotions
when you become a professor.
- There is hence a great pressure to publish as many papers as possible, and
to thus break your research into Minimal Publishable Units (MPUs), each
of which is a separate paper. Some people also take things by first
publishing an MPU as a conference paper (usually omitting details such
as proofs) and then publishing it again as a journal paper (this time
with details).
- While the MPU approach is sometimes seen as necessary, MPUs do not get a lot of
respect and if over-used my harm your academic reputation -- hence, their use
should be balanced with taking longer to create papers that tell complete stories
wrt a research problem (the Shannon approach).
- What makes an MPU varies with academic discipline and subdiscipline, and is best
figured out by chatting with your supervisor and other researchers in your area.
- Once you have written a paper, the natural question is, where do you publish it?
This is the topic that will take up the remainder of our course lectures.
- Presenting Research II: The Joy of Conferences
- Benefits: Publishing at conferences is not only a great way to get
quick feedback on your research (especially if it is still in
progress) but it also gives you a chance to meet and converse with
other researchers (which can be useful in many ways, e.g.,
getting advice on research, meeting potential research
collaborators, furthering research with existing collaborators,
establishing or maintaining your presence in your research community).
- Drawbacks: Conference travel costs money and can take time away from
the research process and conference publications, while valid
CV entries (especially if the conference is refereed), are not valued
as highly as journal papers or chapters in edited books.
- There are typically a number of conferences -- how do you choose
which one is best for you and your work? There are a variety of
reasons for choosing a conference, e.g.,
- conference topic (the conference topic is either the same as
your work or a topic you want to know more about)
- conference attendees (the conference is attended by people
you know or want to know)
- conference location (the conference is somewhere close by (and
hence cheap to travel to) or somewhere you want to go
(possibly for non-academic reasons))
- conference date (the conference fits in with other travel you
are doing (so you can get more bang for your travel
dollar) or the conference proceedings will appear in time for
you to cite any accepted paper(s) in an upcoming grant application)
- conference acceptance standards (the conference accepts few or
lots of papers)
There is typically more than one reason for picking a particular
conference, and it is quite acceptable for you to weight reasons
according to your priorities when making a decision.
- An excellent source for conference candidates is the list of
conferences that published papers that you cite in your work;
another is your graduate supervisor.
- Where possible, publish in reputable conferences.
- Conferences whose submissions are refereed are to be preferred
over those that are not.
- Conferences that publish proceedings in accepted series,
e.g. Lecture Notes in Computer Science (Springer), are
to be preferred over conferences that self-publish proceedings.
- Conferences sponsored by established Computer Science
organizations, e.g., ACM, IEEE, are to preferred over
conferences that are not.
- Once you have selected a conference, download its typesetting
template immediately and use it to prepare your paper from as
early as possible; this prevents last-minute formatting problems
from derailing your submission.
- Even if your paper is accepted, there may be varying grades of
acceptance, e.g.,
- Paper accepted for oral presentation with publication of
full paper in proceedings.
- Paper accepted for poster presentation with publication of
full paper in proceedings.
- Paper accepted for poster presentation with publication of
abstract in proceedings.
As posters and abstracts are not as valued as talks and full
papers, think seriously about whether it is worthwhile to attend
if you are offered the second or third alternatives.
- You can list accepted conference papers in your CV (and some
grant applications) before the conference proceedings is
published -- mark them as "to appear", e.g.,
Wareham, T. (2016) Some notes on conference submission. To appear,
16th International Conference Conference (ICC 2016).
As soon as the proceedings are published, fill in the full details in
your CV accordingly.
- Book conference travel and hotels as early as possible to make
sure you get what you want and need, e.g., travel dates and/or
hotels close to t6he conference site, for a reasonable price.
- If a hotel is held in a conference, you sometimes get a reasonable rate
as a conference attendee; if not, consider staying at a
reasonably-priced (and possibly better quality) hotel nearby.
- Apply for student travel grants if the conference offers
them -- these help defray costs (leaving money for other
conference travel) and can also be nice CV entries in
themselves.
- Conference attendance guidelines
- If you are a graduate student, travel if possible with your
supervisor, especially if your supervisor knows people at
the conference -- this is a great way to get introduced to
people, e.g., my experiences with Bill Day and Bill
Gasarch. If you cannot travel with your supervisor, see if
they can arrange for you to meet some of their contacts at
the conference.
- You do not have to attend everything scheduled at the
conference -- look at the schedule before the conference and
at the end of each conference-day to ensure that you attend the
events that are most useful to you (and have backup events in
place if something that looks interesting turns out to be a dud).
This is particularly crucial at large conferences with
parallel event-tracks.
- Lots of stuff gets done at conferences outside of scheduled
events, e.g., around snack-break tables, at meals;
stay alert for such opportunities and balance them up against
the scheduled events.
- By all means network but don't be pushy about it -- some people,
e.g., myself, do not react well to being obviously
(even if gently) networked.
Wednesday, March 15
- Student Presentations II
- Speakers for today:
- Osunrinde, Dayo
- Dowzell, Jake
- Beheshti, Majid
Monday, March 20
- Student Presentations II
- Speakers for today:
- Albira, Rida
- Huang, Xin
- Alfosool, Ali
Monday, April 3 (Lecture #10)
- Guest speaker: Dr. Matthew Hamilton (Doing Industrial Research)
[Slides]
- Presenting Research III: The Joy of Journals
- Benefits: Journal papers are highly-valued contributions to
your research community and CV and are the accepted venue for
distributing completed research.
- Drawbacks: Journal publication can take a long time (especially if
the paper is rejected by one or more journals) and there may be
unanticipated monetary costs, e.g., open access fees.
- There are typically a number of journals -- how do you choose
which one is best for you and your work? There are a variety of
reasons for choosing a journal, e.g.,
- journal topic (the journal topic is the same as that of
your paper)
- journal readership (the journal is read by people who you
want your work to be read by)
- journal publication costs (if any)
- journal acceptance standards (the conference accepts few or
lots of papers)
- journal review and publication timeline (the journal promises
quick review and publication of papers)
There is typically more than one reason for picking a particular
journal, and it is quite acceptable for you to weight reasons
according to your priorities when making a decision.
- An excellent source for journal candidates is the list of
journals that published papers that you cite in your work;
another is your graduate supervisor.
- Where possible, publish in reputable journals.
- Journals sponsored by established Computer Science organizations,
e.g., ACM, IEEE, and/or journal publishers are to
preferred over journals that are not.
- Measurements of journal importance like impact factors can also
be used to judge journal reputability.
- Once you have selected a journal, download its typesetting
template immediately and use it to prepare your paper from as
early as possible; this prevents last-minute formatting problems
that may hold up submission (or publication if the paper is
accepted).
- Once your paper is accepted, make your revisions and submit the
final version as quickly as possible -- to do otherwise can
dramatically prolong the time to publication.
- Example: The publication history of van Rooij and
Wareham (2012).
- You can list journal papers in your CV (and some grant applications)
when the papers have been submitted, when they are being revised, and
when they have been accepted but not yet published. There are a
variety of ways of doing this format-wise, e.g.,
Wareham, T. (2016) Some notes on conference submission. Submitted
to The Journal of Conferences.
Wareham, T. (2016) Some notes on conference submission. Under
revision for The Journal of Conferences.
Wareham, T. (to appear) Some notes on conference submission.
The Journal of Conferences.
As soon as the paper is officially published, fill in the full details
in your CV accordingly.
References
Created: November 10, 2016
Last Modified: March 14, 2017
