Skip to content
Nadeem Akhtar

Research

I work on efficient large language models: parameter-efficient fine-tuning, federated learning, model compression, and reasoning evaluation. Two papers are accepted (in press) at Springer LNNS venues, two are under review, and one is in preparation.

GitHub github.com/NadeemAkhtar1947 Hugging Face huggingface.co/nadeem172 ORCID orcid.org/0009-0000-7293-2163

Research threads

2026

Federated fine-tuning when clients use different LoRA ranks

In federated fine-tuning, phones and servers often train LoRA adapters of different sizes. The usual fix pads the smaller ones with zeros, which wastes bandwidth. NSVD-Agg merges adapters with a nested SVD instead, so no padding is sent.

  • 53.1% less communication, with lower final loss, across 7 seeds on Llama-3.2-3B and Qwen2.5-3B (Wilcoxon p = 0.0156; Cohen's d = 5.76–7.89).
  • A stationarity bound under non-convex assumptions, with a four-term error decomposition that isolates rank-induced projection error.

Paper: Beyond Zero-Padding: Nested-SVD Aggregation for Heterogeneous-Rank Federated LoRA

2026

Does compression break reasoning, not just answers?

A compressed model can reach the right answer through wrong steps. This study checks each reasoning step, not only the final answer, after 4-bit quantization and 50% pruning.

  • About 49,000 generations across 7 model families and 6 compression methods.
  • A step-level annotation schema that separates error propagation, recovery, and compensating-error masking, validated with two human annotators and an LLM judge.

Paper: Beyond Answer Accuracy: Step-Level Evaluation of Compressed Small Language Models

2025–2026

Reasoning across languages in diffusion language models

Diffusion language models generate text differently from standard left-to-right models. This work tests whether their reasoning holds up when the same maths problems are asked in Hindi.

  • A 722-item Hindi/English dataset derived from GSM8K, with syntactically scrambled variants, all checked by hand by a fluent Hindi speaker.
  • Dream-v0-Instruct-7B against Qwen2.5-7B-Instruct: a 90.3-point reasoning-elicitation collapse, from English (91.6%) to Hindi (1.3%), tested with exact McNemar tests.

Paper: Cross-Lingual Reasoning Elicitation Gaps in Diffusion Language Models

Publications

Accepted

  1. Akhtar, M. N. and Singh, A.

    FedQuant: Mixed-Precision Federated Fine-tuning of Large Language Models under Android Hardware Heterogeneity

    Accepted PCCDA 2026 Sejong University, Seoul · Springer LNNS (Scopus-indexed) In press

    Hugging Face

Under review

  1. Akhtar, M. N. et al.

    Beyond Zero-Padding: Nested-SVD Aggregation for Heterogeneous-Rank Federated LoRA

    Under review ICLR 2027

  2. Akhtar, M. N. et al.

    Beyond Answer Accuracy: Step-Level Evaluation of Compressed Small Language Models

    Under review ARR October 2026 cycle

In preparation

  1. Akhtar, M. N. et al.

    Cross-Lingual Reasoning Elicitation Gaps in Diffusion Language Models

    In preparation Targeting ACL 2027

Models and datasets

Public models and datasets are on Hugging Face at huggingface.co/nadeem172.

FedQuant

For FedQuant: Mixed-Precision Federated Fine-tuning of Large Language Models under Android Hardware Heterogeneity

Models

Datasets

Indian legal NLP

For Fine-Tuning Large Language Models for Indian Legal NLP: A Multi-Task Comparative Study

Models

Datasets