{"id":1740,"date":"2024-08-16T06:13:03","date_gmt":"2024-08-16T06:13:03","guid":{"rendered":"https:\/\/icter.lk\/?page_id=1740"},"modified":"2024-12-04T09:10:39","modified_gmt":"2024-12-04T09:10:39","slug":"ucsc-iit-workshop-on-shared-task-towards-building-a-sinhala-tamil-large-language-model","status":"publish","type":"page","link":"https:\/\/icter.lk\/icter_2024\/workshops\/ucsc-iit-workshop-on-shared-task-towards-building-a-sinhala-tamil-large-language-model\/","title":{"rendered":"UCSC – IIT Workshop on Shared Task towards building a Sinhala\/Tamil Large Language Model"},"content":{"rendered":"

[vc_row][vc_column][vc_column_text]Recent advancements in Large Language Models (LLMs) have transformed the world by facilitating access to information. Building a Large Language Model (LLM) is a significant and ambitious project. Unfortunately, the lack of an LLM for the Low resourced languages like Sinhala and Tamil presents\u00a0 significant challenges. Hence there is a huge requirement for developing LLM models for low resourced languages. This can be achieved through the following two approaches.<\/span><\/p>\n

    \n
  1. Building an LLM from scratch, which includes main subtasks like tokenization, embedding, training and fine tuning, in addition to acquiring a substantial amount of text data.<\/span><\/li>\n
  2. Developing a language interface to an open-source LLM using Retrieval-Augmented Generation (RAG).<\/span><\/li>\n<\/ol>\n

    In parallel to the ICTer 2024 conference, and as a follow-up of the Shared Task on Sinhala LLM at iCIIT 2024, UCSC and IIT are collaboratively\u00a0organizing a workshop titled \u201cShared Task towards building a Sinhala\/Tamil Large Language Model<\/strong>\u201d<\/em> which includes two main tasks.<\/p>\n

    Objectives<\/b><\/h3>\n
      \n
    • To advance the development of a Sinhala\/Tamil LLM.<\/span><\/li>\n
    • To foster collaboration and innovation in the NLP community.<\/span><\/li>\n
    • To create robust benchmarks for Sinhala\/Tamil NLP.<\/span><\/li>\n<\/ul>\n

      [\/vc_column_text][vc_custom_heading text=”Shared Task 01″ font_container=”tag:h1|text_align:left” use_theme_fonts=”yes”][vc_column_text]<\/p>\n

      Developing a Tokenizer and a Word Embedding Model for Sinhala\/Tamil Language<\/b><\/h2>\n

      This task involves developing an efficient tokenizer and a robust word embedding model specifically tailored for the Sinhala and Tamil languages.\u00a0<\/span>[\/vc_column_text][vc_message message_box_style=”3d” message_box_color=”orange” icon_fontawesome=”fas fa-paper-plane”]<\/p>\n

      Subtask 01: <\/i><\/b>Developing a Tokenizer for Sinhala\/Tamil<\/b><\/h4>\n

      Tokenization is a fundamental step in natural language processing (NLP) where text is segmented into smaller units, such as words or subwords, which can then be processed by language models. Generic tokenizers are often designed based on linguistic rules and structures prevalent in widely spoken languages like English. Since Sinhala and Tamil languages have unique linguistic characteristics, using generic tokenizers for LLM development\u00a0 is not efficient. Therefore, it is important to have a proper tokenizer for both Sinhala and Tamil languages that is suitable for LLM development. In this task, our goal is to build an efficient tokenizer for Sinhala\/Tamil Language.<\/p>\n

      Objective<\/b><\/h5>\n

      To develop and evaluate tokenization algorithms specifically tailored for Sinhala\/Tamil, considering their unique scripts and linguistic properties.<\/span>[\/vc_message][vc_message message_box_style=”3d” message_box_color=”orange” icon_fontawesome=”fas fa-paper-plane”]<\/p>\n

      Subtask 02:<\/i><\/b> Developing a Sinhala\/Tamil Word Embedding Model (Word2Vec)<\/b><\/h4>\n

      The word embedding model will transform the above tokens into dense vector representations that capture their semantic meaning, which is crucial for various NLP tasks such as translation, summarization, and sentiment analysis. <\/span>This task involves developing <\/span>a Word Embedding model that will transform <\/span>the above tokens into dense vector representations, capturing their semantic meaning. This process involves creating a numerical representation of words in a continuous vector space, where words with similar meanings are mapped to similar vectors.\u00a0<\/span><\/p>\n

      Objective<\/b><\/h5>\n

      To create word and sentence embedding models for Sinhala\/Tamil that capture semantic and syntactic nuances.<\/span>[\/vc_message][vc_custom_heading text=”Shared Task 02″ font_container=”tag:h1|text_align:left” use_theme_fonts=”yes”][vc_column_text]<\/p>\n

      Developing a Chatbot System with Q&A Support Using the RAG Model for Sinhala\/Tamil<\/b><\/h2>\n

      The second task involves participants utilizing available open-source Large Language Models (LLMs) to develop a chatbot system capable of providing Q&A support using the Retrieval-Augmented Generation (RAG) model. Participants will be provided with an initial dataset and a set of questions that the chatbot should be able to answer. The evaluation of the chatbot systems will be conducted separately to ensure a fair and comprehensive assessment.<\/span><\/p>\n

      Task Objectives:<\/b><\/p>\n

        \n
      • Foundation:<\/b> Participating teams should use existing open-source LLMs as the foundation for their chatbot systems.<\/span><\/li>\n
      • RAG Model:<\/b> The RAG model, which combines retrieval-based and generation-based approaches, should be used to enhance the chatbot’s ability to provide accurate and contextually relevant answers.<\/span><\/li>\n
      • Vector Databases:<\/b> Open-source vector databases will be utilized to efficiently store and retrieve vector representations of text data.<\/span><\/li>\n
      • Accuracy and Efficiency:<\/b> The chatbot should be able to answer the given questions accurately and efficiently, demonstrating the effectiveness of the RAG model in Q&A tasks.<\/span><\/li>\n<\/ul>\n

        [\/vc_column_text][vc_message message_box_style=”3d” message_box_color=”orange” icon_fontawesome=”fas fa-paper-plane”]<\/p>\n

        Subtask 01<\/i><\/b>: Developing a Retrieval-Augmented Generation (RAG) model for Sinhala\/Tamil<\/b><\/h4>\n

        Implement and fine-tune RAG models for Sinhala\/Tamil, combining retrieval mechanisms with generation capabilities.<\/span>[\/vc_message][vc_message message_box_style=”3d” message_box_color=”orange” icon_fontawesome=”fas fa-paper-plane”]<\/p>\n

        Subtask 02<\/i><\/b>: Sinhala\/Tamil Interface development<\/b><\/h4>\n

        Develop a user-friendly interface for interacting with the LLM in Sinhala\/Tamil, making it accessible for end-users and developers.<\/span>[\/vc_message][\/vc_column][\/vc_row][vc_row][vc_column][vc_column_text]<\/p>\n

        NOTE<\/b><\/h4>\n
          \n
        • An initial dataset and a set of questions will be provided to train and test the chatbot systems.<\/span><\/i><\/li>\n
        • The evaluation metrics will be disclosed in due course.<\/span><\/i><\/li>\n<\/ul>\n

          [\/vc_column_text][vc_message message_box_style=”3d” message_box_color=”success” icon_fontawesome=”far fa-calendar-alt”]<\/p>\n

          Timeline <\/b>(For both Shared Tasks<\/em>)<\/h3>\n
            \n
          • Call for registration: <\/span> September 06th, 2024<\/strong><\/del><\/li>\n
          • Initial briefing: <\/span> September 10th, 2024<\/strong><\/del><\/li>\n
          • Interim progress meeting (Online): <\/span> September 30th, 2024<\/strong><\/del><\/li>\n
          • Final Submission: November <\/strong><\/span>01st, 2024\u00a0<\/strong><\/li>\n
          • Shared Task competition: <\/span> November 8th, 2024<\/strong><\/li>\n<\/ul>\n

            [\/vc_message][vc_column_text]<\/p>\n

            Who Can Participate?<\/b><\/h3>\n

            Teams from Universities and Industries are Invited to Compete!<\/p>\n

              \n
            • Each team can include up to 5 members and can participate for free.<\/li>\n
            • Other interested parties can join (8th November) by registering with a fee of 5000 rupees.<\/li>\n<\/ul>\n

              Don’t miss this exciting opportunity to learn and collaborate at our conference workshop. Register now!<\/p>\n

              Organized By<\/b><\/h3>\n

              [\/vc_column_text][\/vc_column][\/vc_row][vc_row][vc_column width=”1\/2″][vc_custom_heading text=”Language Technology Research Lab, UCSC” font_container=”tag:h4|text_align:center” use_theme_fonts=”yes”][vc_row_inner][vc_column_inner width=”1\/2″][vc_single_image image=”1805″ img_size=”full” alignment=”center” onclick=”custom_link” img_link_target=”_blank” link=”https:\/\/ucsc.cmb.ac.lk\/”][\/vc_column_inner][vc_column_inner width=”1\/2″][vc_single_image image=”1804″ img_size=”full” alignment=”center” onclick=”custom_link” img_link_target=”_blank” link=”https:\/\/ucsc.cmb.ac.lk\/language-technology-research-laboratory\/”][\/vc_column_inner][\/vc_row_inner][\/vc_column][vc_column width=”1\/2″][vc_custom_heading text=”Informatics Institute of Technology” font_container=”tag:h4|text_align:center” use_theme_fonts=”yes”][vc_single_image image=”1776″ img_size=”full” alignment=”center” onclick=”custom_link” img_link_target=”_blank” link=”https:\/\/www.iit.ac.lk\/”][\/vc_column][\/vc_row][vc_row disable_element=”yes”][vc_column][vc_column_text][vc_btn title=”Click here to register for the competition” shape=”round” color=”warning” i_icon_fontawesome=”fas fa-sign-in-alt” add_icon=”true” link=”url:https%3A%2F%2Fwww.icter.lk%2Fworkshop-registration%2Fpublic%2Fcompetition|target:_blank”][\/vc_column_text][\/vc_column][\/vc_row]<\/p>\n","protected":false},"excerpt":{"rendered":"

              [vc_row][vc_column][vc_column_text]Recent advancements in Large Language Models (LLMs) have transformed the world by facilitating access to information. Building a Large Language Model (LLM) is a significant and ambitious project. Unfortunately, the lack of an LLM for the Low resourced languages like Sinhala and Tamil presents\u00a0 significant challenges. Hence there is a huge requirement for developing LLM […]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":400,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"pgc_sgb_lightbox_settings":"","footnotes":""},"class_list":["post-1740","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/icter.lk\/icter_2024\/wp-json\/wp\/v2\/pages\/1740","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/icter.lk\/icter_2024\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/icter.lk\/icter_2024\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/icter.lk\/icter_2024\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/icter.lk\/icter_2024\/wp-json\/wp\/v2\/comments?post=1740"}],"version-history":[{"count":49,"href":"https:\/\/icter.lk\/icter_2024\/wp-json\/wp\/v2\/pages\/1740\/revisions"}],"predecessor-version":[{"id":2103,"href":"https:\/\/icter.lk\/icter_2024\/wp-json\/wp\/v2\/pages\/1740\/revisions\/2103"}],"up":[{"embeddable":true,"href":"https:\/\/icter.lk\/icter_2024\/wp-json\/wp\/v2\/pages\/400"}],"wp:attachment":[{"href":"https:\/\/icter.lk\/icter_2024\/wp-json\/wp\/v2\/media?parent=1740"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}