I believe that success in this field is not easy and requires a good professor-student relationship, as well as a common research interest.

Major focus in the coming 3-5 years

My research priority for the next 3-5 years will be on building a system for retrieval-based generation and question answering (or data analytics) from multi-modal data lakes, which you can learn more from the Symphony paper (https://www.cidrdb.org/cidr2023/papers/p51-chen.pdf) and its sister system VerifAI (https://arxiv.org/abs/2307.02796) – both are vision papers.

It is challenging to index and retrieve data sets from millions (or billions) of data sets with different modalities (tables, images, graphs, tex files) for natural language based data analytical queries Q. Naturally, there are many open problems, such as:

  • How to index millions of tables, such that given a query Q, we can effectively identify table(s) that can answer Q?
  • How to index millions of tables and text files, such that given a query Q, we can effectively identify text file(s) and table(s) that can answer Q? Moreover, shall we do it in one shot or through iterative retrieval?
  • Can we index thousands of news (image + text) such that we can ask questions such as in how many different NBA players that Obama has played basketball with, and who they are?
  • Ultimately, we want to index all multi-modal data in one system, and use these data to answer any given natural language questions. You can image this is a data lake, with databases, PDF files, excel sheets, emails, and so on, owned by any large corporation.
  • This is hard for both the index part (for example, how to encode different modality of data and align their encodings), and the retrieval part (for example, for each iterative search, we may get more context that may help the next round of retrieval, how to do this and when to terminate?

I have given some hints about research directions. Please try to identify concrete research problems. It is encouraged that you can define the problem based on your background. For example, if you are from the database or NLP background, you may consider “table + text” problems; and if you are from computer vision background, you can consider “image + text” problems. This is to test whether you can identify important problems that combines LLMs and multi-modal data lakes.

Other research interests

I will keep working on the following directions:

  • Data-centric AI
  • AI-powered data preparation
  • LLM-powered data visualization and augmented reality

If you would like to propose ideas on the above directions, please read my papers on these topics firsts, and make sure I can understand and guide you on your proposal. In other words, if you propose something that is out of my scope of knowledge and ability, I cannot help even if it is good.

Application Process

Unless you have a very strong publication record or a very strong recommendation letter, all applicants will follow the below process:

  1. Send me your CV, transcripts and contacts for reference letters. Note: these letters must be confidential that are sent directly to me from your referees.
  2. If you are shortlisted, I will schedule an interview with you. During the interview, please prepare slides to present your background, expertise, and representative research or internship projects. Next step in the interview is to prepare your thoughts or your proposed ideas in the above directions, either my main focus or other research interests.
  3. If the proposed ideas are sound and innovative, I would require each applicant to work on it for ~3 months, either remotely or doing an internship at HKUST (GZ). The main purpose is see whether you like the project and research style.
  4. I will recruit 2-3 PhDs each year from the above candidates. I will send offers in no later than the end of February.

My allocated time for meeting Ph.D. applications/candidates is 3-5pm, every Monday. If this is your first time interview, please book two consecutive time slots, i.e, for one hour. If this is for a periodical discussion, please book only one time slot, i.e., for 30 minutes.