Analysis of the reliability of the evaluation and scoring process for abstract submissions for the “Journées Francophones de la Kinésithérapie 2023”


  • Jean-Philippe Deneuville PRISMATIC Lab, Poitiers UniversityHospital; Institu Pprime UPR 3346, CNRS - Université de Poitiers
  • Matthieu Gallou-Guyot Ochanomizu University, Department Center for Interdisciplinary AI and Data Science, Tokyo, Japon ; Laboratoire HAVAE, Université deLimoges, Limoges, France
  • Matthieu Guémann 1-École Universitaire de Kinésithérapie du centre val de Loire -EUK-CVL, Université d'Orléans, France. 2-Sport, Physical Activity, Rehabilitation and Movement for Performance and Health (SAPRéM), Université d’Orléans, Orléans, France



Reliability, Internal consistency, Clinimetry, Conference abstracts assessment



The "Société Française de Physiothérapie" (SFP) organizes biennial conferences known as the "Journée Francophone de la Rééducation" (JFK) since 2007. The 2023 JFK's scientific committee invites researchers in rehabilitation and reeducation to submit their work prior to the conference. Submissions are evaluated by two reviewers using a predefined rating grid, determining acceptance for presentation. However, the reliability of this process has never been evaluated.


 This study aims to assess the reliability and internal consistency of the JFK submission rating process conducted by the scientific committee.


 Each submission was evaluated blindly by two reviewers employing a standardized 47-item rating grid. The items were categorized into domains like methodology, relevance to physiotherapy, etc. Reliability was gauged using the Intraclass Coefficient Correlation (ICC) for final scores, Cohen's Kappa for individual grid items, and Cronbach Alpha for domain consistency.


The reliability, measured by ICC, was poor (0.003 [CI95 = -0.003 ; 0.888]). None of the individual grid items' Cohen's Kappa exceeded 0.6. Out of 10 domains, 9 had a Cronbach Alpha above 0.7, indicating good consistency. However, 5 domains had an Alpha above 0.9, suggesting redundancy.


 The JFK 2023 submission rating process displayed poor reliability. These findings can guide improvements for the JFK 2025 scientific committee, facilitating an enhanced assessment process.


Additional Files





Original Research


Similar Articles

You may also start an advanced similarity search for this article.