Evaluations
Run models against your data
Introducing Evaluations, a powerful feature designed to enable you to effortlessly test and compare a selection of AI models against your datasets.
Whether you're fine-tuning models or evaluating performance metrics, Oxen evaluations simplifies the process, allowing you to quickly and easily run prompts through an entire dataset.
Once you're happy with the results, output the resulting dataset to a new file, another branch, or directly as a new commit.
Write 20k Unit Tests
b886ddd6-b501-4db8-8ed6-0b719d0ac595 20000 rows completed
Bessie
3 days ago
Prompt: You are a pragmatic Rust programmer. Given the following question, and rust function, write three unit tests for the function. The tests should be a simple line delimited list of assert! or assert_eq! statements.
For example, if the function is:
```rust
fn add_nums(x: i32, y: i32) -> i32 {
x + y
}
```
The unit tests should be
```rust
assert_eq!(add_nums(1, 2), 3);
assert_eq!(add_nums(10, 2), 12);
assert_eq!(add_nums(-10, 2), 8);
```
Make the tests simple and easy to understand. The code should pass `cargo build` and `cargo clippy` and `cargo test`. Do not add a main function or any other code. Respond with only the assert statements and nothing else.
============
Question:
{rust_prompt}
Code:
{rust_code}
Unit Tests:
text → text
Fireworks AI/Qwen 2.5 Coder 32B Instruct
Fireworks AI/Qwen 2.5 Coder 32B InstructSource:
main
Target:
main
Write 20k Rust Code Snippets
febc562a-9bd4-4e91-88d7-a95ee676a5ed 20000 rows completed
Bessie
4 days ago
Prompt: You are a pragmatic Rust programmer. Given the following question, write a Rust function to complete the task. Make the code simple and easy to understand. The code should pass `cargo build` and `cargo clippy`. Do not add a main function. Try to limit library usage to the standard library std. Respond with only the Rust function and nothing else.
Question:
{rust_prompt}
Code:
text → textFireworks AI/Qwen 2.5 Coder 32B Instruct
Fireworks AI/Qwen 2.5 Coder 32B InstructSource:
main
Target:
main
Translate 20k questions to rust
ce45630c-d9e8-4fac-9b41-2d41692076b3 20000 rows completed
Bessie
5 days ago
Prompt: The following prompt may have Python syntax, variable types, and pythonic behavior mentioned. Translate the question to the appropriate Rust syntax and replace anything mentioning Python with Rust.
Do NOT use or mention:
* &'static str
* <&dyn Any>
* lifetimes
* Advanced rust objects
* External libraries
Make sure to use:
* Simple types, i32, f32, String, etc
Only return the translated question and do not answer the question.
Python Prompt:
{prompt}
Rust Prompt:
text → textFireworks AI/Qwen 2.5 Coder 32B Instruct
Fireworks AI/Qwen 2.5 Coder 32B InstructSource:
main
Target:
main
Write Unit Tests
5de81b3b-6ff4-4a8f-8e30-fac2ad240035 200 rows completed
Bessie
5 days ago
Prompt: You are a pragmatic Rust programmer. Given the following question, and rust function, write three unit tests for the function. The tests should be a simple line delimited list of assert statements.
For example, if the function is:
```rust
fn add_nums(x: i32, y: i32) -> i32 {
x + y
}
```
The unit tests should be
```rust
assert_eq!(add_nums(1, 2), 3);
assert_eq!(add_nums(10, 2), 12);
assert_eq!(add_nums(-10, 2), 8);
```
Make the tests simple and easy to understand. The code should pass `cargo build` and `cargo clippy` and `cargo test`. Do not add a main function or any other code. Respond with only the assert! statements and nothing else.
============
Question:
{rust_prompt}
Code:
{rust_code}
Unit Tests:
text → textFireworks AI/Qwen 2.5 Coder 32B Instruct
Fireworks AI/Qwen 2.5 Coder 32B InstructSource:
Target:
Write Rust Code
d0ae4b63-88dd-4d71-aecc-c94c16cb1904 200 rows completed
Bessie
5 days ago
Prompt: You are a pragmatic Rust programmer. Given the following question, write a Rust function to complete the task. Make the code simple and easy to understand. The code should pass `cargo build` and `cargo clippy`. Do not add a main function. Respond with only the rust function and nothing else.
Question:
{rust_prompt}
text → textFireworks AI/Qwen 2.5 Coder 32B Instruct
Fireworks AI/Qwen 2.5 Coder 32B InstructSource:
Target:
Write Rust Code
4604d487-0c60-4fae-a2cc-b0960d498ca6 123 / 200 rows cancelledcancelled
Bessie
5 days ago
Prompt: You are a pragmatic Rust programmer. Given the following question, write a Rust function to complete the task. Make the code simple and easy to understand.Do not add a main function. Respond with only the rust function and nothing else.
Question:
{rust_prompt}
text → textFireworks AI/Qwen 2.5 Coder 32B Instruct
Fireworks AI/Qwen 2.5 Coder 32B InstructSource:
Target:
92236742-81fb-4a76-be7d-8c8883f57647
92236742-81fb-4a76-be7d-8c8883f57647 5 row sample completed
Bessie
5 days ago
Prompt: You are a pragmatic Rust programmer. Given the following question, write a Rust function to complete the task. Make the code simple and easy to understand.Do not add a main function. Respond with only the rust function and nothing else.
Question:
{rust_prompt}
text → text
OpenAI/GPT 4o mini
OpenAI/GPT 4o miniSource:
Translate question to rust
a7c13788-8fc9-4b71-86ae-dae3acdbc2dc 200 rows completed
Bessie
5 days ago
Prompt: The following prompt may have python syntax, variable types, and pythonic behavior mentioned. Translate the question to the appropriate rust syntax and replace anything mentioning python with rust.
Only return the translated question and do not answer the question.
Prompt:
{prompt}
Rust Version:
text → textFireworks AI/Qwen 2.5 Coder 32B Instruct
Fireworks AI/Qwen 2.5 Coder 32B InstructSource:
main
Target:
Translate Tests with DeepSeek-v3
bd51c545-df0a-43e9-bdc1-c61426b5598d 5 row sample completed
Bessie
5 days ago
Prompt: You are a programmer refactoring Python to Rust. You are given a problem statement, rust code, and example tests in python. Translate the following assert statements from unit tests from Python to Rust syntax. Be careful and translate the functionality of the code clearly. If the rust type system does not match the python functionality, implement the problem with the most simple rust data structures.
Only respond with the assert statements and nothing else.
Problem Statement:
{text}
Rust Code:
{rust_code}
Python Tests:
{test_list}
Rust Tests:
text → textFireworks AI/Deepseek V3
Fireworks AI/Deepseek V3Source:
Translate Tests with Qwen 2.5 Coder 32B Instruct
2b0f721f-155c-4c97-8835-559539e0957e 90 rows completed
Bessie
5 days ago
Prompt: You are a programmer refactoring Python to Rust. Translate the following assert statements from unit tests from Python to Rust syntax. Be careful and translate the functionality of the code clearly. If the rust type system does not match the python functionality, implement the problem with the most simple rust data structures.
Only respond with the assert statements and nothing else.
Python Tests:
{test_list}
Rust Tests:
text → textFireworks AI/Qwen 2.5 Coder 32B Instruct
Fireworks AI/Qwen 2.5 Coder 32B InstructSource:
Target:
qwen_coder
DeepSeek-v3 Translating Python -> Rust
bbc4cc1d-28fb-4553-8f28-a03157d08f8a 90 rows completed
Bessie
5 days ago
Prompt: You are a programmer refactoring python to rust. Given the following description for what the code does, translate the code from python to rust. Be careful and translate the functionality of the code clearly. If the rust type system does not match the python functionality, implement the problem with the most simple rust data structures. Do not add a main function. Respond with only the code and nothing else.
Description:
{text}
Python Code:
{code}
Rust Code:
text → textFireworks AI/Deepseek V3
Fireworks AI/Deepseek V3Source:
Target:
Qwen 2.5 Coder 32B Instruct Translating Python -> Rust (save file not in dir)
0f85a3fa-0492-453c-9e5f-4dcae5797aa5 90 rows completed
Bessie
6 days ago
Prompt: You are a programmer refactoring python to rust. Given the following description for what the code does, translate the code from python to rust. Be careful and translate the functionality of the code clearly. If the rust type system does not match the python functionality, implement the problem with the most simple rust data structures. Do not add a main function. Respond with only the code and nothing else.
Description:
{text}
Python Code:
{code}
Rust Code:
text → textFireworks AI/Qwen 2.5 Coder 32B Instruct
Fireworks AI/Qwen 2.5 Coder 32B InstructSource:
Target:
Qwen 2.5 Coder 32B Instruct Translating Python -> Rust
76e494e0-7faf-4b32-bee1-066e0b4ebada 90 rows completed
Bessie
6 days ago
Prompt: You are a programmer refactoring python to rust. Given the following description for what the code does, translate the code from python to rust. Be careful and translate the functionality of the code clearly. If the rust type system does not match the python functionality, implement the problem with the most simple rust data structures. Do not add a main function. Respond with only the code and nothing else.
Description:
{text}
Python Code:
{code}
Rust Code:
text → textFireworks AI/Qwen 2.5 Coder 32B Instruct
Fireworks AI/Qwen 2.5 Coder 32B InstructSource:
Target:
Translate tests with DeepSeek
c7715e12-c2ee-4740-ab2b-935e34278d92 90 rows completed
Bessie
1 week ago
Prompt: Translate the following unit tests from Python to Rust syntax. Only respond with the proper Rust syntax with the assert statements.
Python Tests:
{test_list}
Rust Tests:
text → textFireworks AI/Deepseek V3
Fireworks AI/Deepseek V3Source:
Target:
Translate code to rust
38f89a80-3056-4aaf-8fe2-4c465272575a 90 rows completed
Bessie
1 week ago
Prompt: Translate the following code from python to rust. Respond with only the code and nothing else.
Python Code:
{code}
Rust Code:
text → textFireworks AI/Deepseek V3
Fireworks AI/Deepseek V3Source:
Target:
Rewrite text to text_rust
1fa7770b-02ba-4b26-ad25-2dd0fa373c05 90 rows completed
Bessie
1 week ago
Prompt: In the following text, replace the word "python" with "rust" if it exists, and if the text does not mention rust, modify it to specify we want a rust function. Return the modified text and nothing else.
Text:
{text}
text → textFireworks AI/Deepseek V3
Fireworks AI/Deepseek V3Source:
Target: