Commits - Laurence/Arxiv-Dive-RAG | Datasets at Oxen.ai

Search

Data Branches Evaluations

Arxiv-Dive-RAG/Commits

Merge commit a296eef936aa0b0ed681399002f2831e into 345341eb12e08788708f81c9895d74bc

Bessie Oxingtoncommitted 3 months ago

#78307ca4035bfaeb547c3928483b7153

Bessie Oxingtoncommitted 3 months ago

#345341eb12e08788708f81c9895d74bc

remove oxen columns from committed data/answerable_questions.parquet

Bessie Oxingtoncommitted 4 months ago

#a2022f5e9380532617fb4f255310d25d

filter down to answerable questions

Bessie Oxingtoncommitted 4 months ago

#2a630b989a83016f787969719c0de80b

Filtering questions that are not able to be answered by the context.

Oxcommitted 4 months ago

#819f370dba5e8704d302eb72a7de15fd

add generated data/questions.parquet

Bessie Oxingtoncommitted 4 months ago

#28fe82b1e249cd55806006873acba159

fix parsing of image tags in markdown docs

Bessie Oxingtoncommitted 4 months ago

#88b2a05d377660d2833ac32efd09a2b

update generate_questions script to take in new fields

Bessie Oxingtoncommitted 4 months ago

#42fe37541ff1f96e3269d4ee064e53cf

extract images from pdfs and consolidate title and abstract extraction

Bessie Oxingtoncommitted 4 months ago

#45e2193ebf9eb42905424a0844db7ab

add filter for if the chunks are interesting

Bessie Oxingtoncommitted 4 months ago

#bab824de2330719ec571e714a1b55f23

Bessie Oxingtoncommitted 4 months ago

#d1fc48344755789f88171b099defa359

add script to classify if chunks are interesting

Bessie Oxingtoncommitted 4 months ago

#c4c2890544c48ff5f295163475775a20

add script to generate questions based on the arxiv papers

Bessie Oxingtoncommitted 4 months ago

#69d2888d0fb2bda5d54513f992d3f122

app now collects data from 👍/👎

Bessie Oxingtoncommitted 4 months ago

#529b03e8d1c30b9422a45c0e1b54e255

Updated "title_embedding" with text-embedding-3-small

Bessiecommitted 4 months ago

#a8fc704ed33eff6f0fd060116c3d0ff9