discussion topics

discussion topics

panel topic

question

subtopic

more details

Selected for starter questions

model+training

13

What is the difference between discriminative models and generative models for IR?

Open

x

diffusion models for IR / ranking?

Open

DSI with other architectures: longformer, decoder only (GPT-like), etc.

Open

is LLM-based Retrieval viable: We present sets of document summaries to the LLM, and ask the LLM to determine which documents are relevant + their relevance score.

Open

What is the role of retrieval augmentation?

Open

x

Retrieve then generate VS generate then retrieve

Open

can we retrieve / reconstruct the knowledge base in LLMs

Open

can we RLHF retrieval?

Open

It seems we don’t need RLHF anymore and that simple finetuning works (recent paper I need to find)

Open

agency as a solution to hallucination? e.g. adversarial reactions to decisions linked to false believes

Open

RL with two models: model A has access to a knowledge base and asks questions to model B model B answers based on knowledge stored in its model weights and model A checks the answers the reward is the veracity of model B’s answers.

Open

how can we redesign collective memory indexing geared at an LLM and not at a human

Open

can we train DSI with other LTR objectives than sequence-to-sequence pointwise, such as pairwise and listwise?

Open

model behavior

20

evaluation methodologies — factuality, attribution, diversity, etc.

Open

benchmarks and data sets - what’s missing?

Open

feedback loops - what are the best implicit and explicit means for gathering feedback w/ generative ir systems (esp those that summarize multiple pieces of content into a unified summary)?

Open

completness, exclusivity, relevance ordering

Open

can we quantify how much the model hallucinate on the prompt (”faithfulness”?) VS on implicit knowledge corpus at training time (”factuality”?)?

Open

multimodality as a solution to hallucination?

Open

what is the good term: hallucination / truthfulness / honesty / factuality? Does it matter to agree on a term?

Open

how much can we fit in the prompt

Open

What is the way forward for answer generation? Longer prompts / LLM memory / more of the same / scaling data and params?

Open

x

What’s next for Generative Document Retrieval? Do we need to change docID representations / loss function / make it multimodal / can we implement in practice?

Open

x

how do we handle several relevant documents per query?

Open

how do we handle truths that require hops between documents?

Open

how do we handle different truths?

Open

can we remove all the trivia facts that the LLM learned, so that it can’t hallucinate and only answers given the prompt?

Open

Is there anything we can do to protect against prompt injections?

Open

dynamic corpora - how to handle new documents, document updates, document deletions

Open

how to design approciate docids?

Open

scalability of DSI

Open

how can we express uncertainty estimates? can we use the logits as proxies for factuality?

Open

is factuality related to robustness to changes in the LLM’s temperature (i.e. randomness)?

Open

broader issues

9

how can open source models catch up with closed source ones?

Open

scaling - how to scale up generative ir models w/ quality and cost in mind?

Open

what do vector database offerings offer over the conventional faiss stack?

Open

how will this parallel technical evolution affect the business models?

Open

how to apply generative ir models to real-world scenarios?

Open

ecosystem effects — with more generation, users may click on results less, what does this mean for content creators?

Open

is it okay to remove the human in the loop with end-to-end IR models? e.g. if I look for when was Cleopatra born, is it okay, that I never get to see the original wikipedia page, that I can read, edit and check for mistakes. Instead I am directly given an answer, optionally with an attribution

Open

x

what happens when the docs are already generated? Do we need IR safety? Is GenIR sustainable in the long-term if we are doing GenIR on generated documents? What happens when genIR consumes content generated with genIR?

Open

x

Is generative IR a good fit for applications to professional factual domains like legal, medical, ...?

Open

x

1

Open

⁠

View of Schedule

View of Schedule

Title

Description

Panelists

Start

End

Moderator

discussion topics

Opening

Thursday 9:00 AM

Thursday 9:30 AM

Panel Discussions

Model Training

Thursday 9:30 AM

Thursday 10:30 AM

Coffee Break

Thursday 10:30 AM

Thursday 11:00 AM

Poster Session

Shared with

REML

⁠

and

ReNeuIR

⁠

Thursday 11:00 AM

Thursday 12:30 PM

Lunch

Thursday 12:30 PM

Thursday 1:30 PM

Panel Discussions

Broader Issues

Thursday 1:30 PM

Thursday 2:30 PM

Coffee Break

Thursday 2:30 PM

Thursday 2:45 PM

Panel Discussions

Model Behavior

Thursday 2:45 PM

Thursday 3:45 PM

Coffee Break

Thursday 3:45 PM

Thursday 4:00 PM

Roundtable Discussion

Thursday 4:00 PM

Thursday 4:45 PM

Closing

Thursday 4:45 PM

Thursday 5:00 PM

There are no rows in this table

⁠

warm-up questions

warm-up questions

panel topic

panelists

question

model+training

What is the role of retrieval augmentation?

model+training

What is the difference between discriminative models and generative models for IR?

model behavior

What is the way forward for answer generation? Longer prompts / LLM memory / more of the same / scaling data and params?

model behavior

What’s next for Generative Document Retrieval? Do we need to change docID representations / loss function / make it multimodal / can we implement in practice?

broader issues

is it okay to remove the human in the loop with end-to-end IR models? e.g. if I look for when was Cleopatra born, is it okay, that I never get to see the original wikipedia page, that I can read, edit and check for mistakes. Instead I am directly given an answer, optionally with an attribution

broader issues

what happens when the docs are already generated? Do we need IR safety? Is GenIR sustainable in the long-term if we are doing GenIR on generated documents? What happens when genIR consumes content generated with genIR?

broader issues

Is generative IR a good fit for applications to professional factual domains like legal, medical, ...?

No results from filter

⁠

topics

topics

Name

panel topic

Notes

questions

questions 2

evaluation

model behavior

Open

training

model+training

Open

RL

model+training

Open

hallucination

model behavior

Open

societal impact

broader issues

Open

input

model behavior

Open

output

model behavior

Open

industry

broader issues

Open

GAG

model+training

Open

other

model+training

Open

human-computer interaction

broader issues

Open

architecture

model+training

Open

There are no rows in this table

⁠