By: Shengdong Zhao (polished by ChatGPT) 

Introduction

An outline is akin to the skeleton or sketch of your paper. It allows you to visualize the potential shape of your paper, helping you determine if it has enough contributions and how the narrative should unfold. An outline also serves as your research agenda, assisting you in planning your time and resources effectively.

When to Write an Outline

Typically, an outline should be one of the first things you create when initiating a research project. At times, when exploring a new topic, you may not yet possess adequate knowledge to draft an outline. In such instances, it would be beneficial to spend some time acquainting yourself with the topic before attempting to create an outline. However, don’t delay for too long; as soon as you’ve gained some familiarity, you can start working on an outline. It may appear rough in its initial iterations, but early drafting is vital as it functions as a research agenda. Over time, it evolves and enriches itself. A well-structured outline provides you with a clear roadmap of the next steps and the effort required for each task.

What Should an Outline Include?

Contrary to what many new students believe, an outline is not just an introduction that motivates the paper. The introduction answers a crucial question: Is there a valid reason to pursue this research? This question should be settled before any research projects begin. Therefore, when writing an outline, you should concentrate on outlining the potential research contributions of your paper.

Possible Research Contributions

Different types of papers present diverse avenues for contribution. For an empirical paper, a study that answers a research question can constitute a significant contribution. In the case of an artifact paper, exploring a design space or a comparative study of the proposed solution against baselines and state-of-the-art solutions can be valuable contributions. Depending on your research type, you need to delineate sufficient concrete contributions in your outline to form a solid paper. For a top-tier empirical research paper, you should ideally include at least three studies addressing three key research questions related to your topic. For an artifact paper, the main contribution may come from a novel technique or system that demonstrates significant advantages either by being a pioneer (being the first) or by comparing with the state-of-the-art solution (being the best), summarizing design spaces, or justifying specific design decisions through empirical studies. In your outline, you need to explicitly describe what these contributions are and how they interconnect to form a cohesive paper.

Remember, it is crucial to draft an outline that contains “enough” contributions for a paper as early as possible. Sometimes, you might have some intriguing findings, but they may not suffice for a full paper. In such situations, your priority should be to identify additional research questions or innovative designs to ensure a sufficient number of research contributions before proceeding.

Level of Detail in the Outline

When drafting an outline, steer clear of lengthy paragraphs. Instead, opt for concise sentences and bullet points to clearly demonstrate potential contributions. Bear in mind that an outline is a living document that evolves frequently based on the study’s results or the solution’s design, so avoid spending time refining content that might change. However, it should provide enough details to clearly illustrate your potential contribution (which research question it attempts to answer, any interesting or surprising findings, the content of the design space, the design justifications, etc.). While you may not have the results yet, you can envision potential results, insights, or advantages to highlight the potential contributions.

Time Allocation for Writing an Outline

For each outline, dedicate a few hours to half a day—no more. Once you’ve developed an outline, share it with your collaborators and get feedback. A common mistake is spending days working on an outline without seeking feedback, resulting in wasted time.

Outline Examples 

Empirical Research Study

An outline for empirical research provides a structural overview of a study that aims to answer a research question. It includes key components such as the research questions, study design, tasks, independent and dependent variables, potential outcomes, and termination conditions. The following example is courtesy of Nuwan Janaka. It is important to note that the following represents only one study, which is insufficient for a complete paper. To formulate a comprehensive outline, you likely need to incorporate enough contributions (probably by adding two more studies) to make it a complete outline.

Assumption/vision: Optical head-mounted displays (OHMDs) will be the everyday mobile companion (and will complement/replace mobile phones) that users wear in everyday settings. In such cases, notifications that appear on OHMDs can interrupt people’s daily activities. We are interested in figuring out design features of OHMD notifications that can mitigate interruption to users’ primary tasks. One design feature we are interested in is the animation and timing of notifications. According to attention and interruption research, users can better manage task switching when the secondary notification appears in a gradual fashion instead of instantaneously. The previous results were carried out on desktop computers in stationary settings. Do the same results apply to OHMD in mobile settings?

Background: OHMDs can be used in 2 scenarios. 

• Stationary setting
• Mobile settings

Study 1: Compare OHMD notifications while engaging in a primary task on stationary setting vs mobile settings

• To reduce the interruption of the sudden appearance of OHMD notifications, we focused on fading animation, which has been shown to be effective for desktop notifications. 
• Independent variables (IVs):
  • Primary Task Device (primary task = proofreading)
    • OHMD (sitting)
      • Black/blank computer screen around 1 m in front
    • OHMD (walking)
      • Black/blank computer screen around 1 m in front

• Animation type
  • Fast-Fading (G = 2) : notification comes to full brightness after 2 seconds
  • Slow-Fading 4 (G = 4) : notification comes to full brightness after 4 seconds
  • Instant (G = 0): notifications immediately appear
  • Swipe down with instant (T = 0): notification swipes down from top within 330 ms (current way of presentation)

• Thus our primary research question was, How effective are fading animations in reducing interruption on OHMD notifications?
  • RQ1: How does the fading animation compare to the blast and scrolling animations in terms of task performance and perceived distraction during multitasking? Is there an optimal fading duration?
    • H1: Fading animation reduces the interference to primary tasks compared to instant and scrolling animation.
    • H2: Fading animation is less distracting and cognitively less demanding than instant and scrolling animation.
    • H3: There is an optimal range for fading duration
  • RQ2: Does the effect of fading depend on the primary task context?
    • H4: Interruption of OHMD notifications in mobile walking condition will be lower for the primary task on OHMD than the primary task on stationary condition.
    • H5: Optimal fading duration depends on the primary task context.
• Dependent variables (within-subject)
  • Primary task:
    • Reading time
    • Reading accuracy
    • Adjusted reading accuracy = reading accuracy/reading time (consider the effect of slowing reading to increase accuracy) \cite{}
  • Secondary task:
    • Notification recognition accuracy
    • Noticeability of notifications
    • Understandability of notifications
  • Perceived effects of interruption
    • Perceived interruption
    • Perceived task load (NASA-TLX)

Possible outcomes and potential implications:

There many possible outcomes:  

1. Gradual notification produces less interruptions as compared with existing notification display styles in mobile walking conditions (users are able to read text better and resume reading faster), but is similar to the existing design in stationary settings because XXX.  
   1. Implications: notification needs to be designed differently for OHMD in mobile conditions, which can significantly reduce interruption to users’ primary task. Given the importance of notification in everyday usage, this can be an impactful result. 
2. Gradual notification is no better than existing design in all conditions. 
   1. The result has no practical implications as there is no change needed for existing design, however, this result is different from the literature. If we can find out why it is so and provide an insightful analysis, this can still be a paper that offers new knowledge to the community. 

Artifact Paper Outline

The contributions of an artifact are typically categorized as either ‘be-the-first’ or ‘be-the-best’. Here is an outline for a ‘be-the-best’ menu technique paper, entitled ParaGlassMenu, which discusses subtle interactions during face-to-face conversations, authored by Cai Runze. It’s important to note that given the simplicity of the technique, it doesn’t necessarily include sections on [design space exploration] or [formative study to motivate the design]. The exploration of design space is an aspect that requires iterative development. A formative study, while crucial, is usually conducted once the artifact has proven to be sufficiently intriguing with convincing results from pilot studies.

Introduction

• Interacting with digital information has become a ubiquitous task, even in conversation
  • Interaction Type [mentioned in Related Work]
    • Related to the conversation topic or involved parties
    • Handling personal issues or conversation-irrelevant tasks
  • It brings convenience and positive effects
  • It also brings negative effects, e.g., going against the established social norms, reducing the perceived communication quality, degrading social relationships, and creating misperceptions. Add: distractions (to both users and conversation partners)
• To balance the utility and disruption of digital interaction in social settings, subtle interaction could be the solution
  • Reason (involves 2 definitions of subtle interaction)
    • Non-intrusive to users’ attention -> better focus on conversation  (distraction to users)
    • Hiding the activities -> avoid interruption (distraction to others)
  • Current Approach
    • Attention-maintaining interface
      • Enable non-intrusive notification
      • Not support subtle input while bi-directional interaction is common in daily life
• We proposed ParaGlassMenu
  • Support non-intrusive menu and subtle thumb-index interaction (ring mouse input)
    • Justification in [Related Work]
      • Why non-intrusive menu
        • Help maintain attention
      • Why ring mouse
        • Other inputs either have low accuracy, e.g. foot interaction 
        • Or hard to maintain eye contact with others, e.g. gaze interaction
        • Thumb-index interactions do not influence eye contact and enable high accuracy 
          • Ring mouse is better than other hand-held devices due to easiness of carrying and hiding 
  • Develop IoT applications to evaluate its usage and solve realistic issues users faced in daily life (Provide example)
    • Users in a meeting may want to lower the Air Con speed when feeling cold. With ParaGlassMenu, they can manipulate it secretly to avoid interruption while maintaining attention when others are speaking.
    • Host in conversation wants to check the rice cook’s remaining time before dinner, but it is not necessary to tell the guest. With our tool, the host can do it subtly while maintaining attention on the guest.

‘

Propose the ParaClassMenu Technique and describe its details 

…

Evaluations

• Study 1: In the lab evaluation
  • Comparative study in lab-controlled settings [N=20]
    • Comparative interfaces: Phone, Voice, and Linear Menu
      • Justification [in Related Work]
        • Phone & Voice
          • Current IoT manipulation methods are either hard to support eye contact or hard to hide activities enable opaque manipulation. Thus we need to select representative baselines which cover these two limitations.
          • Previous studies compare subtle interaction with traditional commercial interfaces. 
        • Linear Menu
          • Find the most suitable layout <so there are two RQs?>
    • Task:
      • Check info
      • Control
        • Discrete manipulation
        • Continuous manipulation
        • Select from list
      • Justification
        • Two types of IoT tasks are defined by previous studies: check info and control
        • In the control task, we have three subtasks based on the analysis of google home traits.
    • Measure
      • Quality of conversation
        • [O] Face focus
        • [S] Politeness, Naturalness, perceived workload
      • Quality of IoT manipulation
        • [O] Duration, Accuracy
        • [S] Relaxation, SUS
    • Results:
      • ParaGlassMenu helped users maintain attention on their conversation partner and allowed them to control IoT devices with the highest usability in a socially acceptable manner.
      • Most preferred interfaces
• Study 2: Verify the external validity of ParaGlassMenu (study 1 results) in modeled realistic setting [N=12 pairs]
  • IV: With / Without IoT manipulation
    • No_IoT condition (baseline): measured the conversation’s quality without the usage of ParaGlassMenu and confounding factors due to current limitations with OHMDs, such as weight and appearance
  •  Task:
    • No_IoT condition: Friends Talk, no IoT manipulation (but can adjust the environment before the study)
    • IoT condition: Friends Talk, IoT manipulation
  • Measure
    • Quality of conversation
      • [S] Attention & concentration, Eye contact, Naturalness, Perceived impact of OHMD
    • Quality of IoT manipulation
      • [O] Duration, usage
      • [S] Relaxation, SUS, Interruption, Politeness, Hospitality
  • Results
    • Hosts could manipulate IoT devices in a relaxed and polite manner using the ParaGlassMenu Interface to cater to both the hosts’ and guests’ needs with low interference to conversations
  • Discussion
    • The majority of users didn’t deactivate the menu
      • Keep it is non-intrusive
      • Keep it after starting the “pending” task
        • It suggests auto-deactivation based on the type of task
    • Two visibility of manipulation were found <this is expected, right?>
      • Depends on awareness management
      • To ease the burden of multitasking
    • Limitations
      • Ring mouse sensitivity & click sound
      • OHMD limitations
      • Should support guidance for new users

Overall discussion

• Recap RQ1 & RQ2 based on study results
• TODO: How to generalize the visibility of IoT manipulation to general digital interaction
  • What tasks can be done visibly & what tasks users tend to hide (e.g., checking info could be done subtly, controlling <light> could be done visibly)
  • Depends on social
  • Multi-tasking ability
  •  <provide improvement/options>
  • Cake surprise example
• Enhance the ParaGlassMenu
  • Enhance thumb-index interaction 
  • Support two visibility of manipulation
    • Especially for the  transparent task since our current work mainly design for the hiding manipulation
  • Optimize the breadth & width of the menu
• Extend the ParaGlassMenu to other scenarios where users need to maintain attention on the center target while engaged with digital interactions
• Potential conflict with social norms
  • Whether users will be easily distracted from the conversation if more subtle conversation-irrelevant apps are provided in the future
  • Whether it would be misperceived: misperceived manipulation even though the users didn’t manipulate the device.

Knowing what to focus on

Once your outline has been drafted, it serves as a roadmap to pinpoint your immediate focus. The subsequent steps are often dictated by the most critical uncertainties of your research. So, what constitutes the most critical unknown? Simply put, it is the absence of which leaves you unsure if you have a viable paper.

For example, if you’re working on an empirical paper seeking surprising results, your goal might be to challenge a conventional belief. Let’s assume the prevalent belief is that reducing difficulty enhances user experience. A surprising result, in this case, would be if your findings contradict this belief. Such unexpected findings can form the foundation of an empirical research paper and constitute your main contribution. If you don’t unearth any interesting or surprising results, then that contribution is absent. Therefore, your primary focus should be to test your hypotheses and ascertain whether the results are indeed surprising through initial studies, rather than diverting your attention to other tasks such as implementing techniques or conducting literature reviews.

In other words, your objective is to determine if you have a solid basis for a paper. Everything you do should aim to validate that you indeed have a compelling case for a paper. Until this is confirmed, it is advisable to avoid tasks that aren’t directly related to answering this question.

The terms “believable” and “rough” are particularly noteworthy at this preliminary stage. You simply need to amass sufficient evidence—potentially from a ‘fair’ test with around 4-5 participants—to convince yourself and your collaborators of the potential validity of your findings or design. It’s crucial to avoid the trap of striving for perfection in your study design or overextending your testing group for statistical significance at this point. These efforts are best saved for when you’re certain you have a solid foundation for your paper. If you spend extensive time on pilot studies only to find the results lacking or unexpected, it can lead to wasted resources. Only once you’re confident in your paper’s premise should you invest significant time in refining your study or interface design. This approach ensures your resources effectively contribute to a high-quality paper.

Knowing When to Change Direction

One crucial point in any project is knowing when to cut losses. If the results aren’t promising, consider changing direction or trying a different idea. Consult with your supervisor or seniors if you’re unsure—it’s wiser to switch to a more promising problem than persist with an unpromising one.