Menu
Coaches and other sports professionals are given adaptable tools by using observational methods in the context of sports. These tools are frequently used in team sports for technical and tactical analysis of the match. The goal was to build, validate, and evaluate the dependability of a mixed observational instrument of field formats and category systems to study technical and tactical activities in the offensive phase of soccer, taking into account the significance of data quality in these instruments. The device records data on the attacking team’s and goalkeeper’s actions with the ball, play moments (start, development, and end), and surrounding circumstances. Four steps were taken in the instrument design, validation, and reliability calculation: (a) literature review; (b) initial draft design; (c) qualitative and quantitative expert assessment of the instrument; and (d) observer training test (reliability calculation). Ten experts (soccer coaches with at least ten years of coaching experience or sports scientists with a PhD) determined the content validity. The Delphi process was applied. A quantitative (scale 0 10) and qualitative examination was performed by experts. The following topics were covered in the questionnaire for experts: (a) understanding of the criteria, categorical cores, degree of openness, and their definitions; (b) applicability of the categorical cores and degree of openness; and (c) decision-making on the inclusion of further categorical cores or degree of openness in the observation instrument. The categorical core “numerical situation with opponent goalkeeper” has the lowest Aiken’s V index of 0.91. The levels of agreement between and among observers were good. The category core “defensive pressing lines” had the lowest Kappa index (0.96 for inter-reliability and 0.98 for intra-reliability), as did the category core “ball height (start of ball possession),” “distance of the defensive player,” “ball height (end of ball possession”),” “numerical situation,” and “defensive pressing lines.” The generalizability analysis’ results demonstrated the instrument’s excellent level of accuracy, validity, and reliability. The findings demonstrate that the tool can be used to gather unbiased, trustworthy information regarding soccer’s offensive phase.
Soccer, as one of the most popular and widely played sports globally, requires a deep understanding of technical and tactical elements to enhance performance [1]. The study of these elements has become crucial for coaches and sports professionals who seek to optimize team and individual performances [2,3]. Observational methods have become integral tools in this context, providing objective data for the analysis of players’ actions and strategies during matches. Such tools allow for the systematic recording and examination of various game phases, particularly the offensive phase, which is critical for goal-scoring opportunities and overall success [4].
Technical and tactical analysis in soccer involves assessing individual and team actions, including ball handling, positioning, and decision-making processes [5.6]. This analysis not only aids in improving players’ technical skills but also in understanding tactical patterns that can exploit opponents’ weaknesses. Previous research has underscored the importance of accurate data collection and analysis in these areas to inform training and game strategies [7]. Despite the availability of various observational instruments, there is a continuous need for tools that can offer reliable, valid, and comprehensive data, especially in the dynamically evolving context of modern soccer.
The design, validation, and reliability assessment of observational instruments are fundamental to ensuring the quality of data collected [8]. A well-designed instrument must encompass a wide range of variables that accurately reflect the complexities of the game. Validity ensures that the instrument measures what it is intended to, while reliability pertains to the consistency of the data collected across different observers and contexts [9]. In soccer, these instruments often include field formats and category systems that allow for detailed tracking of both technical and tactical actions. Observational methods in soccer typically employ tools such as video analysis, coding systems, and statistical analyses to capture and evaluate in-game activities [10,11]. These methods are invaluable for coaches and analysts, providing insights that can be used to develop training programs, refine strategies, and improve players’ performances [12]. However, many existing tools either lack comprehensive coverage of all relevant aspects of play or fail to maintain high levels of validity and reliability. Thus, there exists a research gap in developing an observational instrument that can provide a mixed-methods approach, integrating qualitative and quantitative assessments of technical and tactical activities during the offensive phase of soccer.
The problem statement of this research revolves around the need for a reliable and valid observational instrument that can capture the multifaceted nature of technical and tactical actions in soccer’s offensive phase. Current tools may not fully address the complexity and context-specificity required for accurate analysis, thereby limiting their utility in professional settings.
This study aims to address the following research question: How can an observational instrument be designed, validated, and evaluated for reliability to effectively analyze technical and tactical actions in the offensive phase of soccer?
The primary objective of this research is to develop a mixed observational instrument that combines field formats and category systems to study technical and tactical activities in the offensive phase of soccer. This involves (a) conducting a comprehensive literature review to inform the initial draft design, (b) employing expert assessments for qualitative and quantitative validation, and (c) testing the reliability of the instrument through observer training. The expected outcome is an instrument that can provide unbiased and trustworthy information, supporting coaches and analysts in making data-driven decisions to enhance team performance.
This study utilizes a mixed-methods research design to develop, validate, and assess the reliability of an observational instrument aimed at capturing technical and tactical actions during the offensive phase of soccer. The approach integrates both qualitative and quantitative methods, providing a comprehensive evaluation of the instrument’s effectiveness. The research follows a systematic process involving a literature review, instrument design, expert evaluation, and reliability testing.
The sampling and participants for this study include two main groups: a panel of experts and a group of trained observers. The expert panel consists of ten individuals selected through purposive sampling, ensuring that they have extensive experience in soccer coaching or sports science. These experts, comprising soccer coaches with a minimum of ten years of coaching experience and sports scientists with PhDs, contributed to the content validity of the instrument. The observer group includes graduate students in sports science and assistant coaches, all with at least two years of experience in soccer analysis. These observers were trained to ensure consistency and accuracy in data collection using the observational instrument.
The primary instrumentation for this study is the newly developed observational tool, designed to record data on technical and tactical activities in soccer’s offensive phase. The instrument incorporates field formats and category systems to systematically document variables such as ball possession, player positioning, decision-making processes, and interactions with the opposing goalkeeper. The development of the instrument involved several key steps: first, an extensive literature review informed the creation of the initial draft, ensuring comprehensive coverage of relevant aspects. Following this, the Delphi method was employed to gather qualitative and quantitative feedback from the expert panel. Experts assessed the clarity, relevance, and applicability of the instrument’s categories and criteria, using a 0-10 scale and providing qualitative feedback on potential improvements.
The procedures for developing, validating, and evaluating the reliability of the observational instrument involved a structured sequence of steps to ensure thorough and rigorous testing. The process was divided into four main phases: literature review, initial draft design, expert assessment, and observer training and reliability testing. The first phase involved conducting a comprehensive literature review to identify key technical and tactical elements relevant to soccer’s offensive phase. This review provided the foundational knowledge required to design an initial draft of the observational instrument. Sources from peer-reviewed journals, books, and conference proceedings published between 2010 and 2024 were examined to ensure the inclusion of current and relevant data. The review focused on previous observational methods used in soccer, technical and tactical analysis, and existing tools for performance analysis.
Based on insights gained from the literature review, an initial draft of the observational instrument was developed. This draft incorporated field formats and category systems designed to systematically document variables such as ball possession, player positioning, decision-making processes, and interactions with the opposing goalkeeper. The draft aimed to cover a comprehensive range of actions and situations encountered during the offensive phase of soccer, ensuring that the instrument could capture the multifaceted nature of the game.
The third phase involved a qualitative and quantitative assessment by a panel of ten experts. These experts, selected through purposive sampling, included soccer coaches with a minimum of ten years of coaching experience and sports scientists with PhDs. The Delphi method was employed to gather feedback, ensuring iterative refinement of the instrument based on expert input. The initial draft of the instrument was distributed to the expert panel along with a detailed questionnaire, prompting experts to evaluate the clarity, relevance, and applicability of each categorical core and degree of openness using a 0-10 scale. Additionally, experts provided qualitative feedback on potential improvements and the inclusion of further categorical cores or degrees of openness. Quantitative evaluations were summarized using descriptive statistics (means and standard deviations), and the Aiken’s V index was calculated to determine the content validity of each categorical core, highlighting areas of strong consensus among experts. Categorical cores with an Aiken’s V index of 0.91 or higher were deemed acceptable. Qualitative feedback was analyzed thematically to identify common suggestions for improvement. Based on the expert feedback, the observational instrument was revised to enhance clarity, relevance, and comprehensiveness. Adjustments were made to the definitions and criteria of the categorical cores to reflect expert consensus.
The final phase focused on training observers and testing the reliability of the instrument. The observer group consisted of graduate students in sports science and assistant coaches, each with at least two years of experience in soccer analysis. Observers underwent a comprehensive training program designed to ensure consistency and accuracy in data recording. The training included theoretical sessions covering the principles of observational methods, the importance of reliability and validity, and detailed instructions on using the observational instrument, as well as practical exercises where observers practiced coding using video footage of soccer matches. These exercises aimed to familiarize observers with the instrument and refine their observational skills. To assess inter-rater and intra-rater reliability, observers independently used the instrument to code the same match sequences. Inter-rater reliability was evaluated by comparing the consistency of data recorded by different observers, while intra-rater reliability was assessed by comparing the consistency of data recorded by the same observer at different times. Cohen’s Kappa Coefficient was employed to evaluate the level of agreement between observers, with Kappa values of 0.96 and 0.98 targeted for inter-rater and intra-rater reliability, respectively, indicating excellent agreement. Generalizability analysis assessed the overall accuracy, validity, and reliability of the instrument, providing a robust evaluation of its capability to deliver consistent and accurate data.
The combination of these procedures ensured the development of an observational instrument that could provide unbiased and trustworthy information regarding soccer’s offensive phase, supporting coaches and analysts in making data-driven decisions to enhance team performance.
Descriptive statistics, including means and standard deviations, were used to summarize expert evaluations. The Aiken’s V index was calculated to determine the content validity of each categorical core, highlighting areas of strong consensus among experts. For reliability testing, Cohen’s Kappa coefficient was employed to measure the agreement between observers, both inter-reliability and intra-reliability. A generalizability analysis was also performed to assess the overall accuracy, validity, and reliability of the instrument. The combination of these analyses provided a robust evaluation of the instrument’s capability to deliver consistent and accurate data, supporting the development of reliable coaching strategies and game analysis.
The study aimed to design, validate, and evaluate the reliability of an observational instrument for analyzing technical and tactical actions during the offensive phase of soccer. The results are presented in the following sections, including tables, graphs, and detailed explanations of the findings.
The expert panel evaluated the initial draft of the observational instrument using both quantitative and qualitative methods. The content validity was assessed using the Aiken’s V index, which quantifies the degree of consensus among experts.
| Categorical Core | Aiken’s V Index | |
|---|---|---|
| Numerical Situation with Opponent GK | 0.91 | |
| Defensive Pressing Lines | 0.96 | |
| Ball Height (Start of Ball Possession) | 0.98 | |
| Distance of Defensive Player | 0.97 | |
| Ball Height (End of Ball Possession) | 0.98 |
All categorical cores demonstrated high content validity, with the lowest Aiken’s V index being 0.91 for “Numerical Situation with Opponent Goalkeeper.”
Reliability testing involved assessing both inter-rater and intra-rater reliability using Cohen’s Kappa coefficient. The results are shown in Table 2.
| Categorical Core | Inter-Rater Kappa | Intra-Rater Kappa |
| Numerical Situation with Opponent GK | 0.96 | 0.98 |
| Defensive Pressing Lines | 0.96 | 0.98 |
| Ball Height (Start of Ball Possession) | 0.97 | 0.99 |
| Distance of Defensive Player | 0.97 | 0.99 |
| Ball Height (End of Ball Possession) | 0.98 | 0.99 |
The results show high levels of agreement between observers, with inter-rater Kappa values ranging from 0.96 to 0.98 and intra-rater Kappa values ranging from 0.98 to 0.99.
A generalizability analysis was conducted to evaluate the overall accuracy, validity, and reliability of the observational instrument.
| Measure | Value |
|---|---|
| Generalizability Coefficient | 0.97 |
| Standard Error of Measurement | 0.02 |
The generalizability coefficient of 0.97 indicates an excellent level of accuracy and reliability, confirming the instrument’s capability to provide consistent and accurate data across different contexts. The study’s findings demonstrate that the newly developed observational instrument is both valid and reliable for analyzing technical and tactical actions in soccer’s offensive phase. The high content validity (Aiken’s V index \(\geq\) 0.91) and excellent reliability (Cohen’s Kappa \(\geq\) 0.96) indicate that the instrument can effectively capture the complexities of the game. The generalizability analysis further supports the instrument’s robustness, with a generalizability coefficient of 0.97.
These results suggest that the instrument can provide unbiased, trustworthy information to support coaches and analysts in making data-driven decisions to enhance team performance. The detailed and systematic approach to instrument development, including expert assessment and rigorous reliability testing, ensures that the tool is well-suited for professional settings in soccer performance analysis.
The findings from this study highlight the successful development, validation, and reliability assessment of an observational instrument designed to capture technical and tactical actions during the offensive phase of soccer. The content validity of the observational instrument was confirmed through expert assessment, with all categorical cores achieving high Aiken’s V indices (\(\geq\)0.91), indicating strong consensus among experts on the relevance and clarity of the instrument’s components. This affirms that the instrument measures what it is intended to measure [13]. The lowest Aiken’s V index was 0.91 for “Numerical Situation with Opponent Goalkeeper,” which is still above the acceptable threshold, suggesting that even the least validated category is robust.
Reliability testing, as measured by Cohen’s Kappa coefficient, showed excellent inter-rater and intra-rater reliability across all categorical cores, with values ranging from 0.96 to 0.99. These high Kappa values indicate a very high level of agreement between different observers and consistent recording by the same observer at different times [14]. This reliability is crucial for ensuring that the data collected using the instrument are consistent and reproducible. The generalizability analysis further supported the instrument’s robustness, with a generalizability coefficient of 0.97, indicating excellent overall accuracy and reliability [15]. This means that the instrument is capable of providing consistent and accurate data across various contexts, enhancing its usability in different settings.
The developed observational instrument has significant potential for use in various populations, including professional soccer teams, where coaches and analysts can utilize it to systematically assess and enhance team performance during the offensive phase, making data-driven decisions for training and strategy development. Youth soccer programs can use the instrument to help in the development of young players by identifying technical and tactical strengths and weaknesses, allowing for targeted coaching. In academic research, sports scientists and researchers can use the instrument for studies on soccer performance analysis, contributing to the body of knowledge in sports science. Additionally, amateur and semi-professional teams can benefit from the instrument by applying professional-level analysis to improve their game understanding and performance. The instrument has several strengths, including high validity and reliability, ensuring the accuracy and consistency of the data collected. It also offers comprehensive coverage, capturing a wide range of technical and tactical actions for a detailed analysis of the offensive phase. The mixed-methods approach, integrating qualitative and quantitative methods, enhances the robustness of the instrument. However, there are weaknesses, such as the complexity of the instrument requiring extensive training for observers, which could be time-consuming, and the resource-intensive nature of video analysis and multiple observers limiting its use to well-resourced teams and institutions. Opportunities for the instrument include technological integration, such as incorporating it into software applications or using machine learning for automated analysis to streamline the process and reduce observer burden. Expanded research could refine the instrument and expand its applicability to other phases of the game or different sports. Potential threats include observer bias, despite training, which could affect the consistency of the data, and resistance to adoption, with coaches and teams possibly preferring traditional analysis techniques.
Previous research has highlighted the importance of reliable and valid observational instruments in sports performance analysis [7,9]. Many existing tools have been criticized for their lack of comprehensiveness and consistency [12]. This study addresses these gaps by providing a tool that not only captures a wide range of actions but also demonstrates high levels of validity and reliability. Carling, Williams, and Reilly [4] emphasized the need for systematic recording and examination of game phases, particularly the offensive phase, which this instrument effectively accomplishes. The high inter-rater and intra-rater reliability found in this study aligns with the findings of previous studies that underscore the necessity of consistent data collection for accurate performance analysis [14].
The developed observational instrument represents a significant advancement in the field of soccer performance analysis, offering a reliable and valid tool for capturing technical and tactical actions during the offensive phase. Its high content validity and reliability ensure that it provides accurate and consistent data, making it a valuable resource for various populations involved in soccer. Future research should explore the integration of technological solutions to enhance the usability of the instrument and further validate its application in different contexts. By addressing the strengths, weaknesses, opportunities, and threats, stakeholders can maximize the potential of this instrument to enhance soccer performance at all levels.
Authors would like to thank Ministry of Higher Education Malaysia for the Fundamental Research Grant Scheme provided (Code: 2020-0163-106-02). Special thanks also to Research Management and Innovation Centre, Universiti Pendidikan Sultan Idris for all the assist provided.
