Don’t get me wrong, it’s a fantastic time to be doing AI, but it’s more important to stay sane (or even happy?), let’s talk about the latter here.

Disclaimer: I write purely/mostly from an early PhD academic viewpoint, maybe my friends in the industry have differing experiences (or maybe not).

I will try to organize this by what I think are the sources of the AInxiety, and what it means or leads to as an early academic or even anyone doing AI.

Things that trickle down from limited spots

The term ‘limited spots’ here is overloaded. It starts with first competing to get into a CS grad program (you should look at some of the applicants these days), competing for resources (compute, etc) while being in the program, finding internships, and eventually jobs.

One would imagine after spending (at least) half a decade doing a PhD you’d pretty much be guaranteed a fantastic role, but hey, this is not an MBA (only a slight dig). You could argue that this is the known academic path, yes, this is the typical journey, but the sheer number of people doing AI compared to the “decent spots” is not a pretty number, especially in the last 2-3 years.

As a consequence, you feel a need to “stand out”. This translates to being “experienced in publishing at top AI/NLP venues such as XYZ” for internship applications, not even that, this might even be the bar for even getting a PhD admission in the first place.

But wait, isn’t the point of a PhD to become that researcher? It used to be.

So what does this lead to?

• Pressure to publish: More true in the last few years, with everyone publishing X number of A* papers every year, this has certainly become an implicit thing. You need to be “productive” and demonstrate that you’re “it”, by publishing in these elite venues (which I don’t think should be the aim but a consequence of doing interesting research).

Things that trickle down from the rich become richer

What I mean by this is that within the community, there are certainly groups/individuals that have more resources than others, in terms of computing, workforce, etc; (very often this is more than well deserved)! Some people (from the outside) don’t realize this is also true in academia, maybe more so in experimental/compute-intensive research, and especially now in AI. This certainly puts them at an advantage to solve certain problems that other’s cant;

But what does this mean?

• Fear of being scooped: With many low-hanging fruits in AI these days, many of those ideas you might want to pursue might just be grid-searched away by people with immense resources at their disposal. This creates a state of constant pressure to put your work out as soon as possible; which shouldn’t be the ideal state of mind for a researcher to do deep (or meaningful) thinking.

Things that trickle down from the sheer pace/scale

There are things that I can attribute to the pace of things; which in some way ofc comes down from what I’ve already talked about. But I think a recent change in the last few years, could be attributed to the fact that the industry now has an equal if not higher involvement in the AI research community.

For me, this pace leads to the following:

• Impossible to stay up-to-date: I’m pretty sure now there exist lists of lists of daily top-X AI papers, and this might not even be an exaggeration. It’s just impossible to a) even know what new is coming out each day, and b) even read the abstract of that paper. It’s baffling. I feel awful that I can’t even catch up to the latest in my niche, despite training to eventually be an expert.

• Fear of obsolescence: Unfortunately for xAI researchers, I don’t mean obsolescence of humans (sorry folks). I just mean the constant thought of hey, will what I do today be even useful tomorrow because I have seen way too many works become not-sota-anymore within literal hours. Yes, I am looking at you oAI, Anthropic and everyone else that’s benchmark-maxxing. (I understand where you come from, but it still is a reality).

Is it all that bad? (Read: why are you even doing this?)

A very fair question after reading this, is if it feels this way why are you even doing this?
The answer to that is I really enjoy what I do. I find what I do to be extremely meaningful and enjoyable, but is it all I’d like to think about 24x7? Absolutely not!

Despite my not-so-subtle venting, the vent is not directed toward the work itself but the state we find ourselves in as a community.

Then, are we just doomed? Finding Sanity.

I’ve thought about this for a while, and I’ve found a few things (that are in my control), which have helped me find my sanity (my friends would disagree and tell you I’m not).

• Honest conversations with colleagues: The lab/advisor layer is extremely important. Having the space to have honest conversations about your feelings/state with your peers and advisor is one of the most calming experiences in this chaos. It gives a great sense that we’re all in the same boat, and are here to see each other win.

• Venting to friends & family: or writing blogs into the abyss (lol). Just a simple co-existence and vent session with my goofy friends is more often than not a cure to a lot of issues. It helps put things into perspective and makes me realize it’s not that deep. Pro Tip: Having non-CS friends helps a lot, I don’t want to talk about what I do at the dinner table.

• Disconnecting from social media: underrated, or is it idk, but not using twitter for a few weeks helped a lot. I stopped looking at other people’s work and inadvertently comparing them with my own self, which is extremely unhealthy. Regular breaks from social media - works like a charm.

• Finding your place: One thing that’s helped me is finding areas that I can work in at the pace that I’d like to. Believe me, spending time doing this is extremely helpful in the long run. This, coupled with coming to terms with how much time I want to spend doing what I do, and the potential consequences of not hustling 24x7 has helped.

Last but not least, if you think you need to talk to someone and do not have any outlet, please reach out to me.

These ideas have and are intended to make me live life in the present, to live life with a sense of gratefulness where I live with a “this too shall pass” mindset, for good and for bad. These ideas come and go, some I agree with more than others, and some I live through more than others. Let us see how these ideas, and thoughts evolve and how experiences change my perspectives over time.

Be unapologetically you. There’s no point in living life, than to spend every moment being you (whoever you chose to be), otherwise what’s the point? Of course, depending on the situation, you can decide to show a part of who you are instead of the whole, but never try to be someone else just to fit in.

Go into anything and everything with love in your heart. Sounds super cliched, but I really think that the world/people need more love. We often have preconceived notions of people and situations even before we encounter them, and I’ve found that the only valid notion to have while going into anything is to have love to give and cheers to spread.

Focus on the few. I try to eliminate as many variables as possible from my life where I have no control. This allows me to focus my energy and mental bandwidth on things that I really care about; friends, family, and work. Everything else that I feel is not worth my time, I actively look for ways to eliminate (with often uncomfortable conversations).

Be kind to yourself and then to others. This is everyone’s first time learning to be human, people make mistakes – just learn to first be kind to yourself, and then to others. You, your family, your friends, the people who hurt you, all of them are as new to this as you.

Let go (of control, resentment, others) A lot of where I am, and who I am has been a consequence of things that I have not had a lot of control over. Of course, I have tried my best to nudge myself in certain directions that I thought were meaningful, but a lot of who I am is a consequence of things that in reality have been beyond what I could have imagined or controlled. The perspective of not holding on to things, and having a learn and move on mindset does wonders. I have gotten better at returning to baseline when I have a hard time with things, but I still think I could get better at this.

Health is everything. I realize this every time I have a cold lol, and then forget it as quickly as it goes away. Taking care of my physical health, for me going to the gym, has been a game changer for my physical health and mental health. As someone who’d been skinny his whole life, it has given me more than I thought it would.

People matter (and don’t). I love the analogies one of my dear friends gave me. Your life is like a train, where some people will come and get off at different stations, and some will stay on till the end. It’s worth finding the people that are your people, but most importantly realize that this life is your journey and yours alone.

The more I think about what I’ve written, I understand there are a lot of contradictions, and that’s fair, but more importantly, that’s perfectly okay, being a contradiction is not as bad as people claim.

8 Jan 2025

Embrace discomfort. Go out of your comfort zone. If you spend all your time in situations that are predictable, with the same people, in the same location, having the same conversations, you will miss out on perspectives. Discomfort and failure directly leads to growth.

In another attempt to clarify my thinking, dump things that have been on my mind, and look back, I wanted to do a raw-ish reflection of my PhD journey (Year 1). This document should evolve over the years; where I can reflect, share, and even look forward to the coming days, weeks, months, years, and (hopefully) not decades lol.

I’m a textbook overthinker, I start here with the question, “What do I even write about here?”. The right question, as one of my dear friends would say, “What’s on your mind, (not so) young Rohan?”.

Short, messy table of contents; for most of these I’ll try and do a before/after, what did I think of things from the outside (Pre-PhD and what do I think now).

Primer: Brief intro about myself, pre-PhD life

I started undergraduate in CS, and ended up not enjoying the more conventional, at least in my environment, CS culture. Most people around me seemed to be enjoying writing code for the sake of writing code, being “cracked engineers”. I could not relate and felt out of place. Don’t get me wrong, I did/do like writing code, but that never seemed to be my primary motivation. Through pure fate and having conversations, I met a great mentor and found myself working in the space of ML and healthcare. If I look back, it makes a lot of sense. It has to do with the fact that I was born and raised in a family of doctors, and I’d seen a lot of the pain that surrounds it, conditioning me to care about the problem. Over time I realized; I enjoy pure problem-solving combined with a messiah complex (lol). With research, it was a good fit; I enjoyed caring about the problem, having time to think about it, and writing code as a means to do something meaningful about the things I cared about.

Attempting to continue this, I applied to graduate school right out of undergrad, couldn’t get in at first, and found myself working in consulting (the only company I interviewed for). I was initially very skeptical of consulting but I quickly found myself having a lot of fun. Stayed there for longer than I ever thought I would, doing work that made an impact. I made close friends who I cherish and changed my perspective towards life, making me less judgemental. On some days, working 16 hours a day, I got a few white hairs because of the stress. I called it the salt-and-pepper look.

The decision to do a PhD

Okay, so how did I end up from earning $$$$$s in consulting to doing a (-)$ PhD?
I’d always loved research, and given my undergrad experience, I enjoyed the process. I also enjoyed consulting, I love working with people and problem-solving. So I decided to apply to grad school regardless, because why not? I got into grad school. I decided to defer for a year, why? 1) I was going through a tough time in my personal life (relationships etc.) and did not want to be away from my safe space, and 2) I was really having fun at work and was climbing the corporate ladder pretty quickly. My personal life got easier, time heals; but professionally as I climbed the ladder, I saw myself spending less time doing things I liked, and more time doing things I didn’t. It was hard letting go of a cushy job, but I felt very strongly that I should try to pursue my intellectual curiosity again. I reasoned that I could always come back to managing and working with people later (although you can never be certain).

Looking back, why did I decide to do a PhD? I think the best answer I have for this now, is that I wanted to minimize regret in the long term. Regret of not exploring my curiosity, regret of not moving forward, regret of not trying. Although one could argue that I could also regret doing this, and don’t get me wrong there are hints of it when times get hard. But a 40-year-old Rohan would be proud of trying to do more, learn more, and experience more, at a younger age. I did not want to look back after a decade and be like oh, I had a chance to do this, and I didn’t.

I see it clearer now, than before starting, as to why I decided to do this. Hindsight’s 20/20, right? I think trusting your gut helps, chasing things that feel right and spark curiosity is a good heuristic.

Grad School: from the outside vs. I’m in this

Disclaimer: I was out of the academic setup for 2 years before starting a PhD.

Before starting, I had little to no expectations tbh, I did not get a lot of time to think and feel. The entire logistics around moving kept me pretty busy. The last day home was rough, I had some hours before my flight, and I was jittery. My buddies and family came to see me off, sweet.

The expectations I did have were: I’d hopefully be working on cutting-edge stuff, with pretty smart folks, and enjoy the intellectual freedom of academia. A lot of the first year, I spent trying to play catchup (since I was a bit out of date lol), trying to figure out what excites me, and doing a lot more coursework than I thought I’d be doing. One of the things that did take some time to adjust, is the money factor, you don’t make much money when you’re in graduate school, but it does teach you discipline, which isn’t bad. Fiscal responsibility and such.

One thing that I used to hear from people, but only rang true when I was in the midst of things, was only get into a PhD if you’re reaaallyyy convinced that you want to. I think what people mean by this, is that a PhD is a very lonely and solo journey, it’s a lot of time spent with yourself where you set everything for yourself: expectations, outcomes, pace, goals. In most other setups, these things are set by others (good grades in school, high paying job, goals for promotions) whereas in a PhD, you have more freedom than you’re used to. I like to think doing a PhD is close to running a startup. You are the product, you are not just responsible for doing good work (research), but also for all the other aspects: marketing, distribution, collaborations, etc. Some of it could come to you as a virtue of where you are, your group, advisor, and for some you need to find ways to make it happen. I think the analogy holds. This is also something I learned here, unless you are already one of the cool kids, you have to find ways to look cooler (wear a leather jacket lol), so that people will hear what you say.

I spent a lot of time working on designing systems that scale in tech consulting, and I was pleasantly surprised to see that a lot of that is useful in academia, too (if you have the patience). Not just designing technical systems, but also social systems (your peers, collaborators, etc.) You go far when you work with people you trust and are okay doing a bit of the “high effort” work along with “cool stuff”.

Perceptions of success

This aspect has changed considerably for me. I used to have this number in my mind: number of publications, internships, money. I believe a lot of it came from comparison (thief of joyyy) and buying into other’s definitions of academic success. I felt bad for not being from a top undergraduate school, not being a part of a top lab, not having an ICML paper. I remember when I was an undergrad, I told someone that I wanted to submit a top ML paper in NeurIPS (lol) and a Nature paper in my first year of grad school (or similar).

After getting in, I think differently. I do understand the importance of metrics in a world where there needs to be an objective measure for evaluation. But, I strongly believe now that this is secondary and a byproduct of being a little smart with what you choose to work on amongst the things that pique your interest. Success for me now really means working on an exciting project with good people and being proud of having contributed to answering questions that matter to me (and hopefully others too). This isn’t necessarily disconnected from publishing at great venues, but this should be a consequence of doing good research and not a goal. I have become more and more disconnected from the metrics defined by others. I truly believe that doing meaningful work (for you) is much more important than the endless chasing of metrics that are defined by others. Try to enable others in the process too.

A note on everything else: Life et al.

Before moving, I thought I knew myself enough (as I always do) and then I learned more. Not complaining (or am I). I used to broadly think of professional success as the top priority; things have changed. Through the year, I’ve definitely re-thought my criterias for the ideal outcome after a PhD. I’d definitely like to be in a city where I can experience more of life, do the work I love, and be around people that are family.

Location

SO when deciding schools, I did not care much about location, I come from a big city, but the criteria for me to decide schools was based on how good the school was and the fit with the advisor. Amherst is a beautiful city but is a bit too quiet for me, I was used to being around more people. It certainly took some time to adjust, and I still am adjusting. I was completely wrong on this one, and I’d think about this more if I were to go back.

People

The importance of people in my life has only gone up. I was kind of “locked-in” in my undergraduate, often missing out on experiences because I was always working. I got better, and I learned to value people more, but it got even more so after moving. Leaving behind family and decade-long friendships is/was not easy, it still is probably the most difficult thing. Making friends and/or finding partners in grad school is hard, also more so in a smaller city, there are not enough people to meet, but that’s ok! It’s something that I also did not anticipate.

I am blessed to have great colleagues and a fantastic advisor who are hard-working people, but more importantly, are incredibly kind. They make life easier. Over just one year, I went from immediately making close friends, to having pretty much 0 friends for a considerable time, and then finding some of my closest friends in a while. Something that I think I’ve only realized the importance of recently is knowing who takes energy from me, and letting them go. When you’re away from family, your friends are your family. Everything is about people, success is better enjoyed with people, and pain better endured.

The Game

The game is never-ending. You always want that next promotion, that next paper, that next pat on the back. Over time, I’ve come to peace with knowing how much of life I’m willing to sacrifice for my ambition. This helps. Being a super ambitious guy, sometimes my actions/sacrifices do not match my ambition which results in disappointment. I do not think I’m willing to sacrifice all of life’s experiences for years (if not decades) for a certain professional goal, as much as I love what I do. The game never ends, be it industry, climbing the ranks, or academia, again climbing the ranks. One thing that does ring true with me, is that you should try and spend as much time doing things that you enjoy (read: play) and try to minimize the time you spend playing the game.

Imposter Syndrome

This is something that I have struggled with for a while. Well, it goes in cycles of “we’re so back” and “I know nothing”, and maybe that is the point. I suppose over the one year, I have become more comfortable not knowing. I have realized the importance of not waiting too long, and to start work on things when I feel like I have just enough to get started. With the pace of AI research, there are certainly doubts like “How useful are the things that I’m working on now going to be even in 6 months?”. Even though some of what I do might never be “useful”, I certainly have gotten better at figuring things out and enjoying the process (which again is kind of the point of everything).

Et al.

Taking care of your health (emotional and physical) has probably taken the top priority for me. I used to do that before too, but again this has amplified. Working out is probably the daily anchor that I look forward to regardless of whatever is happening around me. Lifting weights helps disconnect, and finding new things/hobbies gives me something to look forward to.

It probably also has something to do with getting older, and the (everyday) quarter-life crisis that comes with it. For now, I’m DJing at house parties, and for friends, music is fun, I enjoy it. I see myself playing gigs at clubs. There used to be days when I used to do hip-hop/breakdancing, maybe I will pick up again. Life is supposed to be on average net positive, and you can decide what your unit of averaging is.

Some platitudes that have started to make more sense

This is probably going to be the most fun thing for me to write, because if you know me well, if any of my friends actually read this, it is that I love professing. And what I mean by that is just giving unwarranted life advice. But I think some of these have started to make more sense than before.

• We know when we let people in, but rarely do we know when to let people go.
• It’s all about asking the right questions.
• Take care of yourself first.
• People are everything.
• Judge less, it’s everyone’s first time doing life.
• Your mind is not always your friend.
• Acceptance is a better word than letting go.
• Meet people where they are, not where you are.
• Tough times build character.
• Be more comfortable not knowing.
• Time helps; this too shall pass.
• Stay present.
• Be unapologetically you.

I wrote a post [Part-1] Beginner Friendly Guide to Getting Started in CS Research discussing a high level overview of the research process in CS (ofc. keeping out the parts where I doomscroll instagram lol). Given the surprisingly positive response, I now attempt to provide a detailed & transparent view of my process; a step by step account of how I do the drills, that are (hopefully) easy to follow along, replicate and eventually improve upon.

Disclaimer: The details discussed here are a representation of my “ideal” process. I have to say that I often find myself skipping certain best practices.
<David Goggins’ voice> Is this just because I’m not disciplined enough? </David>
No (read: yes), but because PhD students like myself often have different commitments: multiple projects, reviewing, teaching, mentoring, fighting the monsters under our bed, you get the idea.

Get Started: Dive into the consumption phase (aka. Literature review)

This document focuses primarily on the “literature review” phase. I prefer the more informal term “consumption” phase; because I believe one should look beyond “reviewing”; to “consume” and end up at a place of understanding (and strong intuition) from where they can start building.

The primary & unsurprisingly common question we tackle here, is “How do I get myself up to speed on a topic?”. What follows is a step-by-step description of how I try to understand the latest and greatest (not necessarily the same) of a particular topic. First, I will outline the main steps, after which we will dive into the details:

1. Step 0 - Finding a topic/area of interest: In this section, I describe my view of how you can find interesting problems around you.
   TLDR: know yourself (& what excites you) and look around.

2. Step 1 - Finding content/papers related to the interest area: This section talks about how you should go around finding information pertaining to the topic you are interested in.
   TLDR: Search & dump.

3. Understanding how things work / connecting the dots: After finding decently relevant pieces of information for the topic of interest, I describe how you should start connecting the dots to develop an intuition for the topic.
   TLDR: Read & organize.

4. Diving deeper into what you like: Once you know the basics, you do what most people are afraid to do: ask questions.
   TLDR: Be the fool, Be obsessed.

Figure 1: High level overview of the consumption phase.

Step 0: Finding a topic or an area that is interesting

Before you start your dalliance with the mystic arts, an interesting question that often precedes is deciding your allegiance (slytherin or gryffindor). In other words, before you start diving into a topic, how do you even find an area or topic within Computer Science that is “interesting”?

1. Know Yourself: I feel this is extremely underrated, especially as a beginner when you are either over or underconfident. You could use this framework to know where you fall, and by spending time observing, you should have a very broad idea of what topic(s) interest/excite you.

   1. Know what inspires you: I have found that there are broadly two ways that people find their areas of interest:

      1. Top-down / Real world-Inspired approach: I see this as taking inspiration from real world problems; finding things that bother you in the world and trying to fix them. I often find inspiration by observing problems this way (for me: it’s healthcare) – and then finding ways in which I can contribute. You could also think of this as a top-down approach, where you observe problems at the highest level of abstraction, and then work your way down to a level where you can provide solutions.

      2. Bottom-up / Tech-Inspired approach: You could also be inspired to solve more technical problems. This could look something like trying to make a recommendation algorithm x% more accurate (i’m already addicted so please no lol), or solving an existing technical problem from a new perspective. A lot of my more theoretical friends fall in this category. This is a more bottom up approach, where you find a specific technical (CS) problem that bugs you and then try to fix it - which in turn contributes to solving multiple problems up the ladder of abstraction.

      Note: These two approaches are not mutually exclusive, but just a broad lens that I like to use.

   2. Know where you are now: What do you know and what do you not know?t? You should know where you are with your current skill set, both technical and conceptual (at least broadly). This helps you manage your own expectations and helps to understand where you can start contributing now.

2. Look Around: A good heuristic (in life) that I use, is to find people/groups who(se work) inspire me. For research, this looks like: Going up to university websites, looking at the different labs, and drawing inspiration from their work (read: themes and key ideas). There’s nothing dishonorable about seeking inspiration from people who are at the best of their games.

General Advice: Do not wait to find the perfect problem (or solution), because it doesn’t exist! Find a problem that is “good enough” and run with it. You can always decide to pursue different things later, but getting your hands dirty is 100% better than inaction. As a “perfectionist” myself, I (still) actively train myself to not devise the perfect plan and solve everything in my head, but to get going when I have enough to get started.

Figure 2: How do you find your starting point?

Step 1: Finding papers that are relevant to my area of interest/problem

As talked about in the previous document, I find interesting research either when I’m, a) doing project/topic specific scoping, or b) through general consumption, i.e twitter, my lab slack, people I know etc. My key focus here is on curating things I find interesting.

Method A: Topic Specific Scoping
In practice, I use the following approach whenever I am trying to read up on a particular pre-identified topic:

1. Search: Go to Google Scholar or similar search engines (semantic scholar) for academic papers.
   1. Search for review/survey papers on the topic. I search for the keywords of the topic + “review/survey paper” on Gscholar. Note: Review papers (if well written) are a great starting point that give a very detailed high level overview of the topic.
   2. (Additionally) I search for the keywords of interest on Gscholar directly. Eg: “LLMs” + “Fairness”, “Language models” + “Fairness” + “Health” to find interesting papers. Note: I limit my search up to a certain page number on scholar search after which the relevance starts reducing (based on title).
2. Dump: I then create a list of these papers that I encounter; I use a combination of zotero and excel.
   1. If I find a decent survey paper: I start generating a list of the recent articles that seem relevant and are cited in the survey paper. If the survey paper is recent enough, it should have the latest and OG work both cited. Caveat: Every field has its own pace, the definition of what is “recent” really depends on the topic/field you pick.

   2. If searching with keywords: I only read the title and the abstract of the paper as my first filter, and dump anything I find interesting into an excel (or any other system).

      Heuristic: One heuristic that I implicitly use (not always) is that I will look at the i) citation count, ii) venue, and iii) authors as a signal for quality papers. (more in FAQ).

At the end of this, you should have a list of papers that you found interesting which follow a common theme or are trying to tackle the problems you found interesting in step 0.

Note: It is indeed difficult to differentiate between good vs not so good research in the beginning, but trust me it gets easier over time. You will learn to see through (most, but not all) of the BS.

Method B: General Consumption
I also encounter a lot of interesting papers on social media (twitter) or my lab’s slack, that I end up bookmarking/saving.

1. Bookmark: I let them simmer in my bookmarks for a while, a lot of them don’t end up being read; and that is ok!.
2. Dump: I have a section in my zotero called “interesting ideas” where almost all papers that seem interesting end up, and are read later.

Figure 3: Finding content that is relevant to your interests.

Step 2: Understanding where the frontier of research is, i.e how is it all connected

Having done some homework and curated your reading list. The next thing you want to ask yourself is, “how do I go from a list of related literature to a broad understanding of (the latest work in) this topic”. Answer? Reading and structuring.

I follow a simple templatic approach to do this, which allows me to get through my (sometimes not so easy) readings from the list curated above:

1. Read:
   1. (If available) Read the review paper first.
   2. Then, sort the list of interesting papers in reverse chronological order, filter the papers based on title and abstract, (or pick an OG paper), and start reading. Note: refer to FAQ on “how to read a paper” and “signals for filtering quality papers”.
2. Organize:
   1. While reading, I start filling my literature review sheet (fill out the important details in an excel) [Template] Literature Review.
      Important: My objective when doing this is not to understand everything 100%, but to understand the topic, landscape and current views on the topic. Limiting myself to just filling this sheet (as best I can) allows me to let go of “perfection” and focus on my objective. This is very underrated imo, it helps me keep going when learning complex topics, which get easier to understand only over time.
   2. (Optional) Create a separate document for each topic – write down whatever you learn about the topic from your readings that you think is useful / might need to refer later. Try to prove to yourself why certain things make sense.

Figure 4: Start connecting the dots by reading and organizing what you read.

Step 3: Dive Deeper

Once you have completed a broad overview of the topic, you should have a high level intuition of what is happening and recognize the “vibes”. The next step is to shift your focus to diving depth first. It’s repeating what we’ve already done but with different objectives in mind:

1. Read fewer papers, but get into more details:
   1. Based on your previous reading and organization, you will have a fairly good idea of the nice vs not-so-nice papers.
   2. Read the important papers and ask yourself more questions about what we know/what is interesting and try to document your thoughts!
      Note: A distinguishing factor between people I find cool in research vs not, are people who question everything (even the assumptions that most people take for granted). Be the fool! Ask questions!
2. Learn as you go: As you go deeper and encounter newer concepts - read them as required (you cannot know everything beforehand). Advice: One mistake I made in the past was to try to know “everything” before moving forward (be it reading concepts, documentations, etc). Over time, I have realized that it’s important to know just enough and keep moving forward. It gets easier only with practice. This does not mean skip the basics tho!

FAQs

1. How to use the quality signals mentioned for filtering papers?

   I use a combination of the following as a filter (mostly implicitly) while figuring out what is worth my time when reading papers:

2. Citation Count: This is a count of the number of times this particular paper has been cited by others. In an ideal world, a paper that is “important” should be cited and used by other works as inspiration. Caveat: People have found a way to rig this; salami slicing (split a simple work into multiple papers) and citation rings (people collectively decide to cite each other’s papers) are a few examples of that.

3. Venue: Certain venues are more selective than others, a knowledge of which venues are considered “top-tier” is also a good heuristic. Caveat: Even peer review is broken (more on that later), but it’s the best system we have.

   There are preprint servers like arXiv and biorxiv which allow authors to put work that has not undergone peer review on the internet, while reading such papers always be skeptical (also be skeptical about peer-reviewed work, too).

   (Not so exhaustive & a very lazy) representation of venues that I like are below:

   1. Conferences: ICLR, ICML, NeurIPS, ACL, NAACL, *ACL, EMNLP,….
   2. Journals: Nature, JAMA, .. (I work in healthcare and ML)

4. How to read a paper?

   As with anything, I would approach this based on objectives. Almost always, I do not want to spend 4 hours trying to understand everything for every paper I read. Before I begin reading, I ask myself what my objective is wrt to the paper, and based on that I usually do the following:

   1. If the topic is new: I will read through the abstract, introduction and related works. This will hopefully give me a good intro to the topic.
   2. If I am aware of the topic: I will read through the abstract, conclusion and skim through the methods to understand the concept broadly.
   3. If I know the topics well / the paper really intrigues me: I will read it more thoroughly i.e go through the methods in more detail and try to work out a strong intuition for the proposed technique.

5. What if I don’t understand something?

   This happens more often than people would like to accept. It really depends on where you are in your research journey,

6. If you’re just starting out: Be comfortable with not knowing a lot of the details. As long as you feel like you’re making progress - keep going. Note: This does not mean ignore the basics, but it means be comfortable not knowing.
7. If you’re deep into a topic/ you’re working on a project, you should:
   1. Talk to a mentor who’s worked in that area for a while (if you’re able!)
   2. If not, refer to more “classical” papers, textbooks, tutorials, and reach out to public communities on discord servers.
      Controversial advice: Learn to say “I don’t know”, and you’ll see that you’ll learn things faster and people will respect you more.

8. You seem to prefer doing a lot of things in sheets over docs, why?

   Having a table structure gives me a good overview of different papers. I find writing in docs useful too, but having everything in one view helps me connect ideas, thoughts etc. It probably also has something to do with my background in consulting, where we looove spreadsheets, for good reason, lol. I think there is a lot of value in structuring and documenting things, it is less common in research than you’d imagine.

9. How do I know if I know enough or should I keep reading/learning?

   At the risk of sounding redundant, whenever I do something, I try to ask myself, what is my intent behind this? When I’m doing a consumption phase or literature review for a new topic, my primary intent is to develop a very high level intuition of the topic (the main idea, usage, pros, cons). If I want to go deeper, I spend more time diving into the nuts and bolts of the topic (read: math and code), the trajectory of the topic (where did it start and where we are now), and the latest cool ideas. It really depends on what your objective is.

10. If I’m just getting started, should I go for what’s hot vs what interests me?

    Broadly: I strongly believe that there is enough space to do meaningful work in almost all areas. Of course, some areas have more space, visibility, and ease of entry than others, but in the long term you should ideally work on your interests, while making the least amount of sacrifices to the high power, that I call “hype”; people use different names.
    Caveat: You should always think about where you are in life, if you are looking for jobs/have bills to pay/people to care for - you should be primarily spending time optimizing for what gets you a job/more money vs exploring your interests. It’s about priorities.

11. What do you mean by an “interesting” research area?

    The good (and bad) thing is that the definition of “interesting” is and should be subjective; it evolves over time, at both, an individual and community level. I think one should consider their personal interest first (which allows one to persist when things don’t work - 99% of the times), and be in touch with reality, in the sense that their interest area should have overlap with what is useful to the broader scientific community (you don’t want to be out there chasing unicorns).

Acknowledgement

This document evolved out of this Miro board (https://miro.com/app/board/uXjVNEw4LMY=/) which I started using for a few intro sessions with people. I would like to thank Casey Maloney for detailed comments and suggestions on an early draft of this document. I would also like to thank Utkarsh Sharma and Kartikeya Chaturvedi for their suggestions.

This document would definitely have elements from other advice/content available, if there’s anything that I have inadvertently copied - please let me know. PS: All opinions are my own.

I am a PhD student at the CS department at the University of Massachusetts Amherst. My primary research interest is making robust and trustworthy language models work in the wild, especially in healthcare settings. Broadly, I care about democratizing access to healthcare and improving healthcare outcomes across the world, and I believe that technology can help us get one step closer.

To begin contributing to scientific research, I had to learn how to leverage limited resources and mentorship. This is a document for my past self, a reflection, that provides a systematic approach to answering the questions: “What is research? Is it for me? and how do I sign up?”.This document takes a first-principles approach to understanding the science of conducting well-grounded research, and hopefully has something for everyone.

Introduction

This is starting as pieces of advice intended to lower the barrier of entry in computer science research. The prerequisites are pretty strict: curiosity and some free time after school (insert sarcasm sign here). It is perfectly ok - if you do not have any prior experience in research, in fact, a fresh perspective might even be better. (If you ask me, we need more unconventional ideas that move beyond passing the “benchmarks” in CS research.)

Note: This is not supposed to be a learn how to code guide. There are existing resources that have already done a better job than I ever could. However, if there is enough interest, I would be happy to compile my thoughts on how I approach “writing code” in CS research — general or for machine learning.

Based on my initial thoughts, I am going to cover a few ideas that are worth articulating (more selfishly to just get them out of my head) and are topics that people often ask me. The current list of topics that I will cover are:

1. Overview of research: An introduction to what doing CS research (with a focus on empirical ML research) looks like from a high-level perspective. I will discuss the components as well as their significance.
2. Layman’s guide to doing research: A step-by-step guide on how I start doing research from scratch. This would incorporate strategies that I use (or have used) and have worked well for me.
   1. Part 2: Consumption Phase has been published! [Part-2] Beginner Friendly Guide to Getting Started in CS Research
3. Fascinating areas: A list of running topics and areas that I find interesting, hopefully also with more specific research questions (in the spirit of open science), in case you would like to collaborate with me or are looking for inspiration.

A “slightly” naive view of doing research: What does CS research look like on a daily basis

When I ask people what they think of computer scientists, I hear one of two responses: either a person sitting behind a computer screen (best imagined with a slouched back) writing code or someone staring at a whiteboard (again, best imagined with frizzled hair) scratching their head in agony. In my (limited) experience, I think research has less to do with physical attributes, although I do admit that some stereotypes are not that far off. Research is a set of basic drills on repeat (read: perseverance), interspersed with moments of glimmer and glum. What does this look like? A LOT of reading, writing ideas & code, running systematic experiments, collaborating with brilliant minds (the best part imo), and above all hoping that you come up with something meaningful to share with the rest of the world.

For ease of explanation and to give you a glimpse into the life of a CS researcher, I have broken down the process into various stages. Note: These stages are not mutually exclusive or exhaustive. In reality, there is a lot of back and forth that takes place, but this simplistic view will give you a quick insight into the game of research.

Level 0: High-Level Overview

If I think about the components of research at the highest level of abstraction that are easy to understand, I would break it down into the following components. (If you want to avoid the storytelling, skip to the diagram directly!)

1. Consumption phase: It all starts with a dire thirst for knowledge (or advisor’s orders) that makes you spend countless hours trying to find the fountain of knowledge (one interesting research idea). In other words, you usually start with consuming, i.e trying to read up as much as possible on a particular topic/field, to understand what is already known by the scientific world. The purpose of this phase is two fold, i) improving your understanding of that area/topic and ii) eventually creating a new piece of knowledge.

2. Ideation phase: With the new found wisdom that comes from drinking the elixir at the fountain of knowledge, you now seek to find the most important problem that could use your great powers. In practice, after consumption, you spend time coming up with an interesting idea or question that is useful for the community (unexplored, underexplored, has significant impact, etc).

3. Experimentation phase: Now that you know you want to solve world hunger, you want to test your brilliant idea in the hope of seeing the world change right before your eyes (and you should probably take a camera because you’d want to record it). Realistically, this is the phase where you design your experiments and try to produce evidence to prove/disprove your idea. Remember: falsifying your hypothesis is also a great result!

4. Presentation phase: Now is your time to shine, your spotlight, where you get all the glory and ofc a red carpet. The reality is a bit less glamorous, usually one spends time trying to package their results so they reach the people who would appreciate and make good use of the findings (a paper).

Figure 1: High-level overview of the research phases.

Level 1: What does each phase look like?

1. Consumption Phase: The consumption phase is really when you try to understand what others know and care about. This serves as your knowledge base for a particular topic or field. For me, it looks like this:
   1. Finding content: Information relevant to you can be from multiple sources (papers, lectures, tutorials), and typically it’s either:
      1. Project-driven: where you already know what topic you want to work on and you want to survey what is currently known.
      2. General consumption: where you come across papers in conferences, and social media, that you find interesting and want to learn more about.
   2. Curation: This usually goes hand in hand with finding content, it involves curating the interesting papers (making a pile of papers).
   3. Filtration: After having a list of papers, you need to filter out which papers are “important”.
   4. Read: Now it’s time you read the selected list of papers. Depending on your goal, you can decide to either quickly skim through or deep dive.

Figure 2: Overview of consumption phase.

1. Ideation Phase: Now that you have read a bunch of stuff, next comes the more fun part (at least for me). After understanding what other people know, you attempt to come up with new ideas.
   1. Listing interesting ideas: This usually involves effort in trying to identify gaps in current work, future work that others have suggested (usually found in the discussion), trying combinations of multiple ideas, etc.
   2. Prioritizing ideas: Based on factors such as relevance to community, ease of implementation, and personal interest - try to pick ideas that check most of your boxes.
   3. Refinement: Once you have a broad idea or question that you want to pursue, you want to spend some time refining it. I usually try to come up with a specific research question, write multiple hypotheses and place bets on what would work (fun to look at retrospectively).

Figure 3: Overview of Ideation phase.

1. Experimentation Phase: Now that you have a “good-enough” research question (which is good enough to get started), or idea that you want to pursue, you wanna get your hands dirty.
   1. Experimental Design: First, given you have a research question and a set of hypotheses, you want to design your study in order to prove/disprove each of your hypotheses. This involves deciding on datasets, experiments, models, metrics etc.
   2. Writing Code: Then you write the code, run the experiments and collect metrics that help you get closer to the truth.
   3. Decipher: Once you get the metrics accumulated, you can start interpreting what those metrics mean and hopefully have something interesting.

Figure 4: Overview of experimentation phase.

1. Presentation Phase: So now that you have solved the most important problem in this world, you want to share your success. This usually is the following:
   1. Structure: Start with creating a structure for the manuscript; write up the heading, titles, and main punchlines.
   2. Dump and Refine: Input the main results, followed by the other sections and refine with your collaborators.
   3. Submission: Lastly, submit it wherever you’d like!

Figure 5: Overview of presentation phase

FAQs

1. Is there any point where you have a full grasp on the prior knowledge of a subject?

   The more time I have spent in research, the more I realize how little I know. Don’t get me wrong, I definitely do understand things at a wider and deeper level as I spend more time, but I am also constantly humbled as I realize that I cannot know it all. For a field like deep learning/LLMs, a lot is still unknown and works on intuition. From a practical perspective (in a world where you need to publish papers), I like the approach of trying to get “just enough” intuition and knowledge of a topic so that i) I do not reinvent the wheel, and ii) I can build knowledge on top. It is very easy to go down the rabbit hole of trying to know everything before starting - but that is near impossible.

2. How do you organize what you’ve learned? or do you just read and think about it?
   1. For papers: I like to keep a mix of 1) papers that I find interesting and want to annotate and read later in zotero, and 2) when doing specific searches for a project, I do that in excel with the objective of doing a literature review. (more details in next blog).
   2. For topic specific knowledge: I usually like to maintain topic specific docs where I write down all I learn / read for a particular topic - helps me refer to it later.

3. How much time does all of this take?

   This really depends on the nature of the project, how exploratory vs pre-decided it is. In my experience, I have had projects that have roughly taken from 6months to 2.5 years, from start to publication. What does this mean? This means that you need to have patience when doing research, often ideas and problem statements evolve, being flexible and having patience are virtuous traits for doing quality research.

4. How much should I hedge my bets? Should I pursue multiple different projects or focus on one topic at a time?

   I would suggest as a beginner (even later) you should focus on one thing at a time. It is often enticing to put yourself in multiple projects in the hope that one of them works. Believe me, there is more value (in the long term) in learning from failures from a single project where you go in depth, than papers from projects where you barely scratched the surface.

Acknowledgement

This document evolved out of this Miro board (https://miro.com/app/board/uXjVNEw4LMY=/) which I started using for a few intro sessions with people. I would like to thank Casey Maloney and Tiye Ngachoko for their comments and suggestions on an early draft of this document.

This document would definitely have elements from other advice/content available, if there’s anything that I have inadvertently copied - please let me know. PS: All opinions are my own.