<?xml version="1.0" encoding="UTF-8"?>
<article article-type="research-article" dtd-version="1.3" xml:lang="ru" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://metafora.rcsi.science/xsd_files/journal3.xsd">
  <front>
    <journal-meta>
      <journal-id journal-id-type="publisher-id">moitvivt</journal-id>
      <journal-title-group>
        <journal-title xml:lang="ru">Моделирование, оптимизация и информационные технологии</journal-title>
        <trans-title-group xml:lang="en">
          <trans-title>Modeling, Optimization and Information Technology</trans-title>
        </trans-title-group>
      </journal-title-group>
      <issn pub-type="epub">2310-6018</issn>
      <publisher>
        <publisher-name>Издательство</publisher-name>
      </publisher>
    </journal-meta>
    <article-meta>
      <article-id pub-id-type="doi">10.26102/2310-6018/2025.50.3.009</article-id>
      <article-id pub-id-type="custom" custom-type="elpub">1967</article-id>
      <title-group>
        <article-title xml:lang="ru">Исследование и оценка качества аннотаций на естественном языке, сгенерированных мультиагентной системой</article-title>
        <trans-title-group xml:lang="en">
          <trans-title>Research and evaluation of the quality of natural language annotations generated by the multi-agent system</trans-title>
        </trans-title-group>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <contrib-id contrib-id-type="orcid">0009-0003-5767-6810</contrib-id>
          <name-alternatives>
            <name name-style="eastern" xml:lang="ru">
              <surname>Кузнецова</surname>
              <given-names>Анна Игоревна</given-names>
            </name>
            <name name-style="western" xml:lang="en">
              <surname>Kuznetsova</surname>
              <given-names>Anna Igorevna</given-names>
            </name>
          </name-alternatives>
          <email>anniekuznec@mail.ru</email>
          <xref ref-type="aff">aff-1</xref>
        </contrib>
        <contrib contrib-type="author">
          <contrib-id contrib-id-type="orcid">0000-0002-0795-2675</contrib-id>
          <name-alternatives>
            <name name-style="eastern" xml:lang="ru">
              <surname>Носкин</surname>
              <given-names>Виктор Викторович</given-names>
            </name>
            <name name-style="western" xml:lang="en">
              <surname>Noskin</surname>
              <given-names>Victor Victorovich</given-names>
            </name>
          </name-alternatives>
          <email>vitek2012rus@gmail.com</email>
          <xref ref-type="aff">aff-2</xref>
        </contrib>
      </contrib-group>
      <aff-alternatives id="aff-1">
        <aff xml:lang="ru">Волгоградский государственный технический университет ООО "ГЛОУБАЙТ АНАЛИТИЧЕСКИЕ РЕШЕНИЯ"</aff>
        <aff xml:lang="en">Volgograd State Technical University OOO "GLOWBYTE ANALYTICAL SOLUTIONS"</aff>
      </aff-alternatives>
      <aff-alternatives id="aff-2">
        <aff xml:lang="ru">Воронежский государственный технический университет</aff>
        <aff xml:lang="en">Voronezh State Technical University</aff>
      </aff-alternatives>
      <pub-date pub-type="epub">
        <day>01</day>
        <month>01</month>
        <year>2026</year>
      </pub-date>
      <volume>1</volume>
      <issue>1</issue>
      <elocation-id>10.26102/2310-6018/2025.50.3.009</elocation-id>
      <permissions>
        <copyright-statement>Copyright © Авторы, 2026</copyright-statement>
        <copyright-year>2026</copyright-year>
        <license license-type="creative-commons-attribution" xlink:href="https://creativecommons.org/licenses/by/4.0/">
          <license-p>This work is licensed under a Creative Commons Attribution 4.0 International License</license-p>
        </license>
      </permissions>
      <self-uri xlink:href="https://moitvivt.ru/ru/journal/article?id=1967"/>
      <abstract xml:lang="ru">
        <p>Исследование посвящено оценке качества аннотаций на русском языке, сгенерированных мультиагентной системой для анализа временных рядов. Система состоит из четырех специализированных независимых агентов: аналитик дашборда, аналитик временного ряда, доменно-специфичный агент и агент для взаимодействия с пользователем. Аннотации генерируются на основе данных дашборда и временного ряда, анализируемых с использованием модели GPT-4o-mini и графа задач для агентов на базе LangGraph. Оценка качества аннотаций проводилась по метрикам понятности, читаемости, контекстуальной уместности и грамотности, а также с использованием адаптированной формулы индекса удобочитаемости Флеша для русского языка. Было разработано тестирование и проведено с участием 21 пользователя на 10 дашбордах – итого 210 оценок по десятибалльной шкале для каждого из показателей. Проведенная оценка и результаты показали эффективность аннотаций: понятность – 8,486, читаемость – 8,705, соответствие контексту – 8,890, грамотность – 8,724. Индекс удобочитаемости составил 33,6, что показывает среднюю сложность текста. Но такой показатель связан со спецификой области исследования и не учитывает расположение слов и их контекст, а только статические показатели длины. Взрослый человек и неспециалист в этой области способен воспринимать сложные слова в аннотации, что доказывают другие оценки. Все оставленные пользователями замечания будут учтены для улучшения формата и интерактивности системы в дальнейшем исследовании.</p>
      </abstract>
      <trans-abstract xml:lang="en">
        <p>This study is devoted to assessing the quality of annotations in Russian generated by a multi-agent system for time series analysis. The system includes four specialized agents: a dashboard analyst, a time series analyst, a domain-specific agent, and an agent for user interaction. Annotations are generated by analyzing dashboard and time series data using the GPT-4o-mini model and a task graph implemented with LangGraph. The quality of the annotations was assessed using the metrics of clarity, readability, contextual relevance, and literacy, as well as using an adapted Flesch readability index formula for the Russian language. Testing was developed and conducted with the participation of 21 users on 10 dashboards – a total of 210 ratings on a ten-point scale for each of the metrics. The assessment and results showed the effectiveness of annotations: clarity - 8.486, readability - 8.705, contextual relevance – 8.890, literacy – 8.724. The readability index was 33.6, which shows the average complexity of the text. This indicator is related to the specifics of the research area and does not take into account the arrangement of words and their context, but only static length indicators. An adult and a non-specialist in each field are able to perceive complex words in the annotation, which is proven by other ratings. All comments left by users will be taken into account to improve the format and interactivity of the system in further research.</p>
      </trans-abstract>
      <kwd-group xml:lang="ru">
        <kwd>временной ряд</kwd>
        <kwd>генерация аннотаций</kwd>
        <kwd>LLM</kwd>
        <kwd>мультиагентная система</kwd>
        <kwd>дашборды</kwd>
      </kwd-group>
      <kwd-group xml:lang="en">
        <kwd>time series</kwd>
        <kwd>annotation generation</kwd>
        <kwd>LLM</kwd>
        <kwd>multi-agent system</kwd>
        <kwd>dashboards</kwd>
      </kwd-group>
      <funding-group>
        <funding-statement xml:lang="ru">Исследование выполнено без спонсорской поддержки.</funding-statement>
        <funding-statement xml:lang="en">The study was performed without external funding.</funding-statement>
      </funding-group>
    </article-meta>
  </front>
  <back>
    <ref-list>
      <title>References</title>
      <ref id="cit1">
        <label>1</label>
        <mixed-citation xml:lang="ru">Gkatzia D., Lemon O., Rieser V. Natural Language Generation Enhances Human Decision-Making with Uncertain Information. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016: Volume 2: Short Papers, 07–12 August 2016, Berlin, Germany. Association for Computational Linguistics; 2016. P. 264–268. https://doi.org/10.18653/v1/P16-2043</mixed-citation>
      </ref>
      <ref id="cit2">
        <label>2</label>
        <mixed-citation xml:lang="ru">Jiang Yu., Pan Z., Zhang X., et al. Empowering Time Series Analysis with Large Language Models: A Survey. In: Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence Survey Track, IJCAI 2024, 03–09 August 2024, Jeju, South Korea. 2024. P. 8095–8103. https://doi.org/10.24963/ijcai.2024/895</mixed-citation>
      </ref>
      <ref id="cit3">
        <label>3</label>
        <mixed-citation xml:lang="ru">Tang F., Ding Yi. Are Large Language Models Useful for Time Series Data Analysis? arXiv. URL: https://doi.org/10.48550/arXiv.2412.12219 [Accessed 15th May 2025].</mixed-citation>
      </ref>
      <ref id="cit4">
        <label>4</label>
        <mixed-citation xml:lang="ru">Jin M., Zhang Yi., Chen W., et al. Position: What Can Large Language Models Tell Us About Time Series Analysis. In: Proceedings of the 41st International Conference on Machine Learning, ICML 2024, 21–27 July 2024, Vienna, Austria. 2024. https://doi.org/10.48550/arXiv.2402.02713</mixed-citation>
      </ref>
      <ref id="cit5">
        <label>5</label>
        <mixed-citation xml:lang="ru">Lin M., Chen Zh., Liu Ya., et al. Decoding Time Series with LLMs: A Multi-Agent Framework for Cross-Domain Annotation. arXiv. URL:  https://doi.org/10.48550/arXiv.2410.17462 [Accessed 15th May 2025].</mixed-citation>
      </ref>
      <ref id="cit6">
        <label>6</label>
        <mixed-citation xml:lang="ru">Sycara K.P. Multiagent Systems. AI Magazine. 1998;19(2):79. https://doi.org/10.1609/aimag.v19i2.1370</mixed-citation>
      </ref>
      <ref id="cit7">
        <label>7</label>
        <mixed-citation xml:lang="ru">Ghylsels E., Osborn D.R. The Econometric Analysis of Seasonal Time Series. New York: Cambridge University Press; 2001. 228 p.</mixed-citation>
      </ref>
      <ref id="cit8">
        <label>8</label>
        <mixed-citation xml:lang="ru">Jebb A.T., Tay L., Wang W., Huang Q. Time Series Analysis for Psychological Research: Examining and Forecasting Change. Frontiers in Psychology. 2015;6. https://doi.org/10.3389/fpsyg.2015.00727</mixed-citation>
      </ref>
      <ref id="cit9">
        <label>9</label>
        <mixed-citation xml:lang="ru">Darban Z.Z., Webb G.I., Pan Sh., Aggarwal Ch., Salehi M. Deep Learning for Time Series Anomaly Detection: A Survey. ACM Computing Surveys. 2024;57(1). https://doi.org/10.1145/3691338</mixed-citation>
      </ref>
      <ref id="cit10">
        <label>10</label>
        <mixed-citation xml:lang="ru">Orgad H., Toker M., Gekhman Z., et al. LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations. In: The Thirteenth International Conference on Learning Representations, ICLR 2025, 24–28 April 2025, Singapore. 2025. https://doi.org/10.48550/arXiv.2410.02707</mixed-citation>
      </ref>
      <ref id="cit11">
        <label>11</label>
        <mixed-citation xml:lang="ru">Štajner S., Evans R., Orasan C., Mitkov R. What Can Readability Measures Really Tell Us About Text Complexity? In: Proceedings of the Eighth International Conference on Language Resources and Evaluation, LREC 2012, 23–25 May 2012, Istanbul, Turkey. European Language Resources Association (ELRA); 2012. P. 14–21.</mixed-citation>
      </ref>
    </ref-list>
    <fn-group>
      <fn fn-type="conflict">
        <p>The authors declare that there are no conflicts of interest present.</p>
      </fn>
    </fn-group>
  </back>
</article>