Olivia (Simin) Fan

  • Ph.D. candidate in Machine Learning at École Polytechnique Fédérale de Lausanne (EPFL), advised by Prof. Martin Jaggi.

  • B.Sc. (honor) in Computer Science at University of Michigan, previously worked with Prof. Rada Mihalcea, Prof. Lu Wang and Prof. Jie Liu.

  • B.Sc. (government honor) in Electrical and Computer Engineering at Shanghai Jiao Tong University.

  • I am mainly working on Skiing, Photography, Piano, Singing, Ballet&Yoga, Badminton, with Leisure time hobby as Machine Learning research ;).

    •    ================================================================

        ❀Welcome to Olivia's WonderLand 🥕
                                          Wish you a nice day! :)

    See my work

    Research Explorations🧐

    My research interests lie in effective and efficient training of large foundation models, especially LLMs, from the following perspectives:
    • Efficient pretraining and post-training by Data Curriculum and Low-rank Modelling strategies;
    • Understanding LLM training dynamics and generalization behaviours;
    • Foundation models for scientific research (AI4Science).

    Publications

    Dynamic Gradient Alignment for Online Data Mixing

    Simin Fan, Pierre Ablin, David Grangier
    [preprint]

    Task-Adaptive Pretrained Language Models via Clustered-Importance Sampling

    David Grangier, Simin Fan, Skyler Seto, Pierre Ablin
    [preprint]

    HyperINF: Unleashing the HyperPower of the Schulz's Method for Data Influence Estimation

    Xinyu Zhou*, Simin Fan*, Martin Jaggi.
    [DMLR Workshop@ICML 2024, preprint]

    MEDITRON-70B: Scaling Medical Pretraining for Large Language Models

    Zeming Chen, Alejandro Hernández Cano, ..., Simin Fan, Martin Jaggi, Antoine Bosselut.
    [preprint]

    DOGE: Domain Reweighting with Generalization Estimation

    Simin Fan, Matteo Pagliardini, Martin Jaggi.
    [ICML 2024]

    Irreducible Curriculum for Language Model Pretraining

    Simin Fan, Martin Jaggi.
    [NeurIPS 2023 Workshop]

    ReadingQuizMaker: A Human-NLP Collaborative System that Supports Instructors to Design High-Quality Reading Quiz Questions

    Xinyi Lu, Simin Fan, Jessica Houghton, Lu Wang, Xu Wang.
    [CHI 2023, Best Paper Nomination]

    Towards Process-Oriented, Modular, and Versatile Question Generation that Meets Educational Needs

    Xu Wang, Simin Fan, Jessica Houghton, Lu Wang.
    [NAACL 2022]

    Genetic Risk Converges on Regulatory Networks Mediating Early Type-2 Diabetes

    Walker JT, Saunders DC, Rai V, Dai C, Orchard P, Hopkirk AL, Reihsmann CV, Tao Y, Fan S, Shrestha S, Varshney A, Wright JJ, Pettway YD, Ventresca C, Agarwala S, Aramandla R, Poffenberger G, Jenkins R, Hart NJ, Greiner DL, Shultz LD, Bottino R, Liu J, Parker SC, Powers AC, Brissova M.
    [Nature 2023]

    Historical OCR Text Quality Analysis and Post-correction

    Instructor: Prof. Sindhu Kutty (UMICH)
    Sponsors: Dr. John Dillon, Dr. Dan Hepp(ProQuest)
    [Multi-disciplinary Project with ProQuest 2022]

       News 🐾


    • [09/2024]  New! Will give a talk at Deep Learning: Classics and Trends (DLCT) on Sept. 20. Save the date if you are interested! 🐶
    • [07/2024]  New! Attending ICML 2024 at Vienna! Reach out to me if you want to chat ☕️
    • [07/2024]  New! One paper on Data Attribution is accepted to DMLR @ICML 2024.
    • [05/2024]  New! One paper on Data Mixture for LLM pretraining is accepted to ICML 2024.