Would love some Feedback on how my Model compares for your Use Cases

#52
by AlioLeuchtmann - opened

https://huggingface.co/AlioLeuchtmann/flash-sqlite its the Pareto Frontier on the Bird Leaderbord in the 2B Class, beating way larger Models
Only trained for Sqlite !

@AlioLeuchtmann strong result for a 2B model. Two checks I would add before comparing it with sqlcoder:

  1. Run it on a second SQL dialect such as PostgreSQL or T-SQL. Models trained only on SQLite often slip on date functions and on LIMIT versus TOP.
  2. Test questions that are not in English.

If Arabic is of interest, our open Gulf Text-to-SQL Bench asks the same 30 business questions in English, MSA and Gulf dialect and scores by execution match: https://huggingface.co/datasets/usmanbilytica/gulf-text-to-sql-bench

Sign up or log in to comment