Evaluating the Consistency of Multiple LLMs with Intra and Inter Reliability Methods

Fadel M. Megahed ¹

Farmer School of Business, Miami University

Ying-Ju (Tessa) Chen ²

Department of Mathematics, University of Dayton

Allison Jones-Farmer ³

Farmer School of Business, Miami University

Gabe Lee ⁴

Farmer School of Business, Miami University

Brooke Wang ⁵

Farmer School of Business, Miami University

Inez M. Zwetsloot ⁶

Faculty of Economics and Business, University of Amsterdam

May 20, 2025

Preface

In this project, we utilize the open-source programming language, alongside Python, to evaluate the consistency of large language models (LLMs) through intra- and inter-reliability methods. For , we are using version 4.4.2 (2024-10-31 ucrt), and we manage package dependencies with renv. The renv package allows for isolated project environments, ensuring the same versions of packages are used consistently. If you’re new to renv, it enables the creation of “snapshots” for package versions, ensuring reproducibility across different systems. You can find all package versions used in this project in our renv repository at this link.

This document is structured using RMarkdown, which seamlessly integrates both and Python code chunks. Readers can tell the difference between and Python code chunks by downloading the raw RMarkdown file from this repository. In addition, the document features a floating table of contents (TOC) on the side, which makes navigation through the sections more convenient. The TOC follows the reader as they scroll, allowing quick access to different sections.

We’ve also implemented foldable code chunks, enabling the user to expand and collapse code as needed to focus on the explanations or results. This feature improves the document’s readability by allowing you to hide the code while reading the main content. If you would like to view or modify the raw code, click the “Code” button at the top of the document to download the full RMarkdown file. This gives you access to all the code chunks used in the analysis.

For the Python sections, we are using a conda environment with Python version 3.12.9. The required Python packages can be installed via the requirements.txt file, available here. The RMarkdown document links to the conda environment, allowing Python chunks to execute alongside the code in the same workflow.

To securely store API secrets for the large language models (LLMs) accessed in the Python chunks, we use a .env file that holds these keys in a safe, environment-specific manner. Similarly, for the stock news API accessed in the sections, we save the API secret in a project-based .Renviron file. Both files keep sensitive credentials secure and allow us to avoid hardcoding API secrets directly in the code. For obvious security reasons, we did not push these .env and .Renviron files to our GitHub repository.

All input and output files for this project can be accessed and downloaded from our GitHub repository.

1 Objectives of this Analysis

We present minimum sample size calculations for our binary classification experiments.
We describe the experimental setup for the consistency analysis of multiple LLMs.
We provide a custom function for LLM classification experiments.
We present an example of a binary classification experiment, using the LLMs to label the sentiment of news articles related to the stock market, and utilize the intra- and inter-rater reliability methods to evaluate the consistency of the LLMs.

2 Sample Size Calculations for our Binary Classification Experiment

The minimum sample size for each of our experiments was computed for: simple percent agreement, Gwet’s AC1 coefficient, and Brennan-Prediger coefficient. The minimum sample size was computed using the tables in Handbook of Inter-Rater Reliability, 5th Edition. Volume 1: Analysis of Categorical Ratings. The sample sizes were computed for the three different metrics, with a margin of error of 0.05, a confidence level of 0.90, and for five replicates.

# create a data frame of the obtained sample size results
binary_data = data.frame(
  metric = c("Percent Agreement", "Gwet’s AC1 Coefficient", "Brennan-Prediger Coefficient"),
  `sample size` = c(216, 1317, 847) |> scales::comma()
)

# generate the table in HTML format
knitr::kable(
  binary_data, format = "html", table.attr = "style='width:50%;'", 
  align = c('l','r')
  ) |>
  kableExtra::kable_styling(full_width = FALSE) |>
  kableExtra::column_spec(1, bold = TRUE)

metric	sample.size
Percent Agreement	216
Gwet’s AC1 Coefficient	1,317
Brennan-Prediger Coefficient	847

Therefore, using the highest sample size among the three metrics, we need at least 1,317 samples for the binary classification experiment.

3 Experimental Setup for the Consistency Analysis

3.1 LLMs Used and their API Keys

In our experiments, we selected the following LLMs:

Closed source proprietary foundation models:

claude-3-7-sonnet-20250219, which is Anthropic’s current best model as of March 12, 2025;
-claude-3-5-haiku-20241022, which represents the smallest model provided by Anthropic as of March 12, 2025;
gpt-4o-2024-11-20, which is OpenAI’s latest snapshot of the 4o series as of March 12, 2025. Note that we do not examine the more recent and capable GPT-4.5 Preview Model as they are very expensive ($150 per million output tokens versus the $10 per million output tokens for the GPT-4o series);
gpt-4o-mini-2024-07-18, which is OpenAI’s latest stable and mini model as of March 12, 2025. These models are fast and are cost optimized for low-latency applications;

Open LLMs

The Open LLMs were run locally using the Ollama interface (v 0.6.0), with the exception of the command-r-plus-08-2024 which we run using the Cohere API as its 104B parameters would be too large to run on our GPU (NVIDIA RTX 5000 Ada Generation, with 32GB of Graphics Memory). The models are:

command-r-plus-08-2024, which is Cohere’s latest large language model as of March 12, 2025;
command-r7b, which is Cohere’s latest small language model as of March 12, 2025. Note that we are running this 7B parameter model locally, via the Ollama interface.;
deepseek-r1:7B, which is a popular model from DeepSeek and is the default when pulling the deepseek-r1 models from Ollama;
deepseek-r1:1.5B, which is the smallest model from DeepSeek;

gemma3:27B, which is the largest open-sourced Gemma3 model from Google, which was released on March 12, 2025;
gemma3:1B, which is the smallest open-sourced Gemma3 model from Google, which was released on March 12, 2025;
llama3.2:3B, Meta went small with its open-sourced Llama3.2 model; this is the largest of the two models that they released as part of the Llama3.2 series;
llama3.2:1B, Meta went small with its open-sourced Llama3.2 model; this is the smallest of the two models that they released as part of the Llama3.2 series;
phi4:latest, Phi-4 is a 14B parameter, state-of-the-art open model from Microsoft.;
phi4-mini, which is the smallest open-sourced Phi Model from Microsoft with 3.8B parameters;

# load the necessary libraries
from dotenv import load_dotenv, find_dotenv
from langchain.prompts.chat import ChatPromptTemplate

# Find the .env path
dotenv_path = find_dotenv()

# load the environment variables
load_dotenv(dotenv_path, override=True)

# the API keys for the different LLMs
openai_api_key = os.getenv('OPENAI_API_KEY')
anthropic_api_key = os.getenv('ANTHROPIC_API_KEY')
cohere_api_key = os.getenv('COHERE_API_KEY')


# the LLM models to be used for labeling
# we ran this over two different run times since we
# had to abort the "exaone-deep" models as they were quite slow
# other commented out models were successfully run in the first run
# so we did not need to run them again
models = [
  # 'claude-3-7-sonnet-20250219',
  # 'claude-3-5-haiku-20241022',
  
  'gpt-4o-2024-11-20',
  'gpt-4o-mini-2024-07-18',
  
  # 'command-r-plus-08-2024',
  # 'command-r7b',
  
  # 'deepseek-r1:7B',
  # 'deepseek-r1:1.5B',
  
  # 'exaone-deep:7.8b',
  # 'exaone-deep:2.4b',
  
  'gemma3:27B',
  'gemma3:1B',
  
  'llama3.2:3B',
  'llama3.2:1B',
  
  'phi4:latest',
  'phi4-mini'
  ]

3.2 A Custom Function for LLM Calls

We created a custom function, generalized_chat_completion, to facilitate the interaction with the LLMs and generate chat completions for each news article in the dataset. The function takes the following parameters:

csv_path: the path to the CSV file containing the news articles;
columns_to_keep: the columns to retain in the final CSV output;
models: a list of chat models to be used;
chat_prompt_template: the prompt template for generating chat messages;
columns_for_chat_prompt: the columns to be used as the user input in the chat prompt template;
num_replicates: the number of replicates per model;
temp: the temperature for the chat model;
max_num_tokens: the maximum number of tokens for chat completions;
save_to_csv: whether to save results to CSV;
output_file: the path to the output CSV file; and
retry_attempts: the number of retry attempts for API errors.

The function reads the CSV file, replicates the data based on the number of models and replicates, sorts the data frame, and iterates through each row to generate chat completions. It uses the specified chat model to generate chat responses, with error handling for API errors. The function saves the results to a CSV file if required and returns the last chat response content if save_to_csv is False.

import os
import time

import pandas as pd
import datetime as dt

from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic
from langchain_cohere import ChatCohere
from langchain_ollama import ChatOllama

def generalized_chat_completion(
    csv_path,
    columns_to_keep,
    models,
    chat_prompt_template,
    columns_for_chat_prompt,
    num_replicates,
    temp=0,
    max_num_tokens=3000,
    save_to_csv=True,
    output_file='../results/generalized_classification.csv',
    retry_attempts=3
):
    """
    Generalized function to generate chat completions from a CSV file, with flexible parameters
    for various chat models, sorting order, and error handling.
    
    Parameters:
        csv_path (str): Path to the CSV file to read data from.
        columns_to_keep (list): Columns to retain in the final CSV output.
        models (list): List of chat models to be used.
        chat_prompt_template (str): The prompt template for generating chat messages.
        columns_for_chat_prompt (list): Columns to be used as the user input in the chat prompt template.
        num_replicates (int): Number of replicates per model.
        temp (float): Temperature for the chat model.
        max_num_tokens (int): Maximum number of tokens for chat completions.
        save_to_csv (bool): Whether to save results to CSV.
        output_file (str): Path to the output CSV file.
        retry_attempts (int): Number of retry attempts for API errors.
    
    Returns:
        None or the last chat response content if save_to_csv is False.
    """
    # read the CSV file
    df = pd.read_csv(csv_path)
    num_rows = df.shape[0]
    total_repeats = len(models) * num_replicates

    # add an index column as 'article_num'
    df['article_num'] = df.index

    # replicate the data frame based on the total repeats
    expanded_df = pd.concat([df] * total_repeats, ignore_index=True)

    # generate model and replicate columns
    model_column = [model for model in models for _ in range(num_replicates * num_rows)]
    replicate_column = [i + 1 for _ in range(len(models)) for i in range(num_replicates) for _ in range(num_rows)]

    # add the model and replicate columns to the data frame
    expanded_df['replicate'] = replicate_column
    expanded_df['chat_model'] = model_column

    # sort the data frame by 'chat_model', 'article_num' and 'replicate'
    expanded_df = expanded_df.sort_values(by=['chat_model','article_num', 'replicate']).reset_index(drop=True)

    # iterate through each row and generate chat completions
    for index in range(expanded_df.shape[0]):
        prompt_data = {col: expanded_df.loc[index, col] for col in columns_for_chat_prompt}
        messages = chat_prompt_template.format_messages(**prompt_data)

        # extract model name and assign the correct chat model
        model = expanded_df.loc[index, 'chat_model']
        chat_model = None
        if model == 'gpt-4o-2024-11-20':
            chat_model = ChatOpenAI(model="gpt-4o-2024-11-20", temperature=temp, max_tokens=max_num_tokens)
        elif model == 'gpt-4o-mini-2024-07-18':
            chat_model = ChatOpenAI(model="gpt-4o-mini-2024-07-18", temperature=temp, max_tokens=max_num_tokens)
        
        elif model == "claude-3-7-sonnet-20250219":
            chat_model = ChatAnthropic(model="claude-3-7-sonnet-20250219", temperature=temp, max_tokens=max_num_tokens)
        elif model == "claude-3-5-haiku-20241022":
            chat_model = ChatAnthropic(model="claude-3-5-haiku-20241022", temperature=temp, max_tokens=max_num_tokens)
        
        elif model == "command-r-plus-08-2024":
            chat_model = ChatCohere(model="command-r-plus-08-2024", temperature=temp, max_tokens=max_num_tokens)
        elif model == "command-r7b":
            chat_model = ChatOllama(model="command-r7b", temperature=temp, max_tokens=max_num_tokens)
        
        elif model == 'deepseek-r1:7B':
            chat_model = ChatOllama(model="deepseek-r1:7B", temperature=temp, max_tokens=max_num_tokens)
        elif model == 'deepseek-r1:1.5B':
            chat_model = ChatOllama(model="deepseek-r1:1.5B", temperature=temp, max_tokens=max_num_tokens)
            
        elif model == 'exaone-deep:7.8b': 
            chat_model = ChatOllama(model="exaone-deep:7.8b", temperature=temp, max_tokens=max_num_tokens)
        
        elif model == 'exaone-deep:2.4b':  
            chat_model = ChatOllama(model="exaone-deep:2.4b", temperature=temp, max_tokens=max_num_tokens)
        
        elif model == 'gemma3:27B':
            chat_model = ChatOllama(model="gemma3:27B", temperature=temp, max_tokens=max_num_tokens)
        elif model == 'gemma3:1B':
            chat_model = ChatOllama(model="gemma3:1B", temperature=temp, max_tokens=max_num_tokens)
        
        elif model == 'llama3.2:3B':
            chat_model = ChatOllama(model="llama3.2:3B", temperature=temp, max_tokens=max_num_tokens)
        elif model == 'llama3.2:1B':
            chat_model = ChatOllama(model="llama3.2:1B", temperature=temp, max_tokens=max_num_tokens)
        
        elif model == 'phi4:latest':
            chat_model = ChatOllama(model="phi4:latest", temperature=temp, max_tokens=max_num_tokens)
        elif model == 'phi4-mini':
            chat_model = ChatOllama(model="phi4-mini", temperature=temp, max_tokens=max_num_tokens)
        
        else:
            print(f"Model {model} is not supported. Skipping this row.")
            continue

        # attempt to generate chat response with retries for error handling
        chat_response_content = "--"
        chat_response_id = None
        for attempt in range(retry_attempts):
            try:
                chat_response = chat_model.invoke(messages)
                chat_response_content = chat_response.content
                chat_response_id = chat_response.id
                break
            except Exception as e:
                error_message = str(e)
                if attempt == retry_attempts - 1:
                    print(f"Failed after {retry_attempts} attempts for model {model} on index {index}. Error: {error_message}")
                else:
                    print(f"Attempt {attempt + 1} failed for model {model} on index {index}. Retrying in 30 seconds...")
                    time.sleep(30)

        # create the row with the desired columns
        data_row = pd.DataFrame({
            **{col: [expanded_df.loc[index, col]] for col in columns_to_keep},
            'chat_model': [model],
            'chat_date': [dt.datetime.now().strftime('%Y-%m-%d %H:%M:%S')],
            'chat_replicate': [expanded_df.loc[index, 'replicate']],
            'chat_response': [chat_response_content],
            'chat_response_id': [chat_response_id]
        })

        # save to CSV if required
        if not save_to_csv:
            continue
        elif not os.path.exists(output_file):
            data_row.to_csv(output_file, index=False)
        else:
            existing_df = pd.read_csv(output_file)
            updated_df = pd.concat([existing_df, data_row], ignore_index=True)
            updated_df.to_csv(output_file, index=False)
    
    if not save_to_csv:
        return chat_response_content

3.3 A Custom Function for Extracting the LLM Labels

We created a custom function, extract_classification, to extract the classification labels from the chat completions generated by the LLMs. The function takes the chat response content and a list of valid words as input. It extracts the classification label from the chat response content based on the valid words provided. The function returns the extracted classification label if it matches any of the valid words; otherwise, it returns NA.

extract_classification <- function(text, valid_words) {
  # ----------------------------
  # Helper: drop rows with unwanted words
  # ----------------------------
  
  drop_row = function(row) { 
    all(is.na(row) | row == "classification.\n\ntemp")
  }
  
  # ----------------------------
  # Helper: Pre-clean text1
  # ----------------------------
  clean_text1 <- function(text) {
    patterns <- c(
      "here ", "", 
      "classification;", "", 
      "definitive classification,", "", 
      "The classification is based on", "", 
      "classification based on", "", 
      "classification based ", "", 
      "classification requires", "it requires", 
      "classification without", "", 
      "it is impossible", "classification: Impossible", 
      "challenging classification", "", 
      "the classification challenging", "it challenging", 
      "\n---\n", " ", 
      "\\*", "", 
      '\\"', "", 
      "cannot definitively classify this news as either Positive or Negative", "", 
      "either Positive or Negative", "", 
      "as:", "as", 
      "\n</analysis>: \n<classification>:", " classification:", 
      "making it challenging to", "", 
      "classification.</classification>", "", 
      "\n\nClassification:\n-", "classification:", 
      "classification.\n\n<classification>:", "classification:", 
      "article", "news", 
      "</analysis>", "<analysis>", 
      "classification\\. ", "", 
      "classification because|classification for", "", 
      "positively", "positive", 
      "negatively", "negative", 
      "leaning slightly towards", "classification:", 
      "the impact is likely to be", "classification:"
    )
    
    for (i in seq(1, length(patterns), by = 2)) {
      text <- gsub(patterns[i], patterns[i + 1], text)
    }
    
    return(text)
  }
  
  # ----------------------------
  # Helper: Normalize sentiment pattern1 to standard format
  # ----------------------------
  normalize_sentiment_phrases1 <- function(text) {
    sentiment_patterns <- c(
      "\\n\\s*<analysis>:",
      "<classification>:(.*?)is likely to be",
      "likely be",
      "is likely to be",
      "is expected to have a",
      "Classification:(.*?)is expected to",
      "classification leans towards",
      "is likely",
      "is likely to have a",
      "leading to a",
      "likely leads to a",
      "suggest a potential",
      "Overall, the immediate"
    )
    
    sentiment_types <- "(positive|negative|neutral|mixed)"
    
    for (pattern in sentiment_patterns) {
      full_pattern <- regex(paste0(pattern, "\\s*", sentiment_types, "\\s*\\n?"), ignore_case = TRUE)
      text <- str_replace_all(
        text,
        full_pattern,
        function(m) {
          sentiment <- str_extract(m, "(?i)positive|negative|neutral|mixed")
          paste0("<classification>: ", str_to_title(sentiment), " ")
        }
      )
    }
    
    return(text)
  }
  # ----------------------------
  # Pre-clean text2
  # ----------------------------
  clean_text2 <- function(text) {
    patterns <- c(
      "\n{1,2}(<classification>)", "\\1", 
      "classification:\n\n<analysis>", "", 
      "the classification becomes challenging", "it becomes challenging", 
      "strict classification guidelines\\.", "", 
      "classification guidelines:|classification guidelines", "classification:", 
      "\n\nClassification:\nThe news has a", " classification:", 
      "classification due to", "", 
      "categorize this news as having either", "to", 
      "categorize this news as|classify this news as|classify this as|the classification should be|still classified it as|I would categorize this as|classified as|is categorized as|making it|making it a", "classification:", 
      "\n</classification>", "", 
      "I classify the impact of this news", "I would classify the impact of this news", 
      "I would classify the news", "I would classify the impact of this news"
    )
    
    for (i in seq(1, length(patterns), by = 2)) {
      text <- gsub(patterns[i], patterns[i + 1], text)
    }
    
    return(text)
  }
  
  # ----------------------------
  # Helper: Match a sentence that starts with the classification phrase and ends with a label
  # ----------------------------
  detect_sentiment_from_classification_sentence <- function(text) {
    match <- str_match(
      text,
      regex(
        "I would classify the impact.*?as\\s+[\"']?(positive|negative|mixed)[\"']?[.?!]?",
        ignore_case = TRUE
      )
    )
    
    if (!is.na(match[2])) {
      output <- paste0("<classification>: ", str_to_title(match[2]), " ")
      return(output)  # Capitalize
    } else {
      return(text)
    }
  }
  # Helper: Normalize sentiment pattern2 to standard format
  # ----------------------------
  normalize_sentiment_phrases2 <- function(text) {
    sentiment_patterns <- c(
      "I would classify this news news as", 
      "the classification is",
      "the tone of the news is",
      "the news presents a"
    )
    
    sentiment_types <- "(positive|negative|neutral|mixed)"
    
    for (pattern in sentiment_patterns) {
      full_pattern <- regex(paste0(pattern, "\\s*", sentiment_types, "\\s*\\n?"), ignore_case = TRUE)
      text <- str_replace_all(
        text,
        full_pattern,
        function(m) {
          sentiment <- str_extract(m, "(?i)positive|negative|neutral|mixed")
          paste0("<classification>: ", str_to_title(sentiment), " ")
        }
      )
    }
    
    return(text)
  }
  
  # ----------------------------
  # Helper: Detect "It is likely that..." sentences
  # ----------------------------
  detect_likely_sentiment <- function(text){
    # Extract all sentences that start with "It is likely that"
    matches <- str_match_all(
      text,
      regex("it is likely that[^.?!]*\\b(positive|negative)\\b[^.?!]*[.?!]", ignore_case = TRUE)
    )[[1]]
    
    if (nrow(matches) > 0) {
      output <- paste0("<classification>: ", str_to_title(matches[1, 2]), " ") # "Positive" or "Negative"
      return(output)
    } else {
      return(text)
    }
  }
  
  # ----------------------------
  # Step 1: Clean and Normalize
  # ----------------------------
  text <- clean_text1(text)
  text <- normalize_sentiment_phrases1(text)
  
  text <- gsub("</classification>:", "<classification>:", text)
  text_temp <- str_extract(text, "(?i)<classification>:\\s*(positive|negative|neutral|mixed)\\s*$")
  if (!is.na(text_temp)) text <- text_temp
  
  text <- clean_text2(text)
  text <- detect_sentiment_from_classification_sentence(text)
  text <- detect_likely_sentiment(text)
  text <- normalize_sentiment_phrases2(text)
  
  text = stringr::str_replace(text, "Mixed to ", "")
  text = stringr::str_replace_all(text, c(
    "Slightly" = "", # Remove the word "slightly"
    "clearly" = "",      # Remove the word "clearly"
    '\\"' = "",          # Remove escaped quotes (\\")
    '"' = ""             # Remove regular quotes (")
  ))
  
  if (str_detect(text, "<classification>: Positive")) text = "<classification>: Positive"
  if (str_detect(text, "<classification>: Negative")) text = "<classification>: Negative"
  if (str_detect(text, "<classification>: Neutral")) text = "<classification>: Neutral"
  
  # ----------------------------
  # Step 2: Extract final classification
  # ----------------------------
  # define a regex pattern to match:
  # 1. "\n\nclassification" with or without special characters or colons after it
  # 2. "\n\n<classification>:" or "\n\n<classification>word</classification>"
  matches = stringr::str_match_all(text, "(?i)(classification[^a-zA-Z]*:?\\s*\\w+|<classification>(\\w+)</classification>)")
  
  # get the last match if it exists
  if (length(matches[[1]]) > 0) {
    if (nrow(matches[[1]]) > 1) {  
      # Apply the function to filter out unwanted rows
      matches[[1]] = matches[[1]][!apply(matches[[1]], 1, drop_row), , drop = FALSE]
    }
    first_match = head(matches[[1]], 1)
    last_match = tail(matches[[1]], 1)
    word = NA
    word1 = NA
    word2 = NA
    # Handle the case for "<classification>word</classification>"
    if (!is.na(first_match[, 3])) {
      word = first_match[, 3]  # Extract word between <classification> and </classification>
    }else if (!is.na(last_match[, 3])) {
      word = last_match[, 3]
    }else {
      word1 = stringr::str_extract(first_match[, 1], "\\w+$")  # Otherwise, extract word after "classification"
      word2 = stringr::str_extract(last_match[, 1], "\\w+$")
    }
    
    # Check if the extracted word is in the valid_words list
    if (tolower(word) %in% tolower(valid_words)) {
      return(word)
    } else if (tolower(word1) %in% tolower(valid_words)){
      return(word1)
    } else if (tolower(word2) %in% tolower(valid_words)){
      return(word2)
    } else {
      return(NA)  # Return NA if no match is found
    }
  } else {
    return(NA)  # Return NA if no match is found
  }
}

3.4 Custom Functions for Computing the Reliability Metrics

# Function to get summary statistics for mean percent agreement
pa_summary = function(df, percent_variable, digits=4){
  llm_models = unique(df$chat_model)
  df_summary = data.frame(model = llm_models, mPa = rep(NA, length(llm_models)))
  
  for (i in 1:length(llm_models)){
    m1 = df |> 
      dplyr::filter(chat_model == llm_models[i]) |>
      dplyr::select(percent_variable)
    colnames(m1) <- "v1"
    
    df_summary[i,2] = paste0(
      round(mean(m1$v1/100), digits), " (", 
      round(sd(m1$v1/100), digits),   ")"
    )
  }
  return(df_summary)
}

# function to compute the other reliability metrics
# should contain one variable called chat_model,
# and vars indicates variables for raters
reliability_coefs = function(df, vars){
  no_models <- n_distinct(df$chat_model)
  df_coefs = data.frame(matrix(NA, ncol=9, nrow=5*no_models))
  
  colnames(df_coefs) = c("model", "coeff.name", "pa", "pe", "coeff.val", "coeff.se", "conf.int", "p.value", "w.name")
  
  llm_models = unique(df$chat_model)
  
  df_coefs$model = rep(llm_models, each=5)
  
  if (is.integer(vars)) vars = colnames(df)[vars]
  
  df_rates = df |>
    dplyr::select(dplyr::all_of(c("chat_model", vars)))
  
  for (i in seq(1, 5*no_models, by=5)){
    m1 = df_rates |>
      dplyr::filter(chat_model == llm_models[ceiling(i/5)]) |>
      dplyr::select(-chat_model)
    df_coefs[i,-1] = irrCAC::conger.kappa.raw(m1)$est
    df_coefs[(i+1),-1] = irrCAC::fleiss.kappa.raw(m1)$est
    df_coefs[(i+2),-1] = irrCAC::gwet.ac1.raw(m1)$est
    df_coefs[(i+3),-1] = irrCAC::bp.coeff.raw(m1)$est
    df_coefs[(i+4),-1] = irrCAC::krippen.alpha.raw(m1)$est
    
  }
  
  return(df_coefs)
  
}

3.5 Custom Functions to Compute Agreement with Benchmark

In the chunk below, we present two functions:

calculate_agreement: This function calculates the agreement between the replicates and the benchmark. It takes the replicates and the benchmark as input and returns the agreement percentage. The function also has an optional parameter na to handle missing values. If na = 0, the function will not consider missing values in the calculation.
ensemble_reps: This function ensembles the replicates by selecting the most common value. It takes the replicates as input and returns the most common value. The function also has an optional parameter na to handle missing values. If na = 0, the function will not consider missing values in the calculation. In case of ties, the function randomly selects one of the most common values.

calculate_agreement = function(reps, ground_truth, na = 0) {
  na_rm = dplyr::if_else(na == 0, T, F, missing = F)
  
  matches = sum(reps == ground_truth, na.rm = na_rm)
  total_reps = length(reps)
  agreement = matches / total_reps
  
  return(agreement)
}

# ensemble function to ensemble the replicates
ensemble_reps = function(reps, na = 0){
  
  # handling of NAs
  na_rm = dplyr::if_else(na == 0, "no", "ifany", missing = "ifany")
  
  # get the most common value
  freq_table = table(reps, useNA = na_rm)
  max_freq = max(freq_table)
  most_common_values = names(freq_table)[freq_table == max_freq]
  
  # randomly select one of the most common values
  most_common = sample(most_common_values, 1)
  
  return(most_common)

}

4 The Binary Classification Experiment

4.1 Extracting the Full Dataset

We use the stocknewsapi to extract articles related to the stock market. We crafted our request to get all tickers from January 05, 2025 to March 13, 2025. The data was pulled on March 14, 2025 at approximately 12:30 EDT. The resulting data frame was saved as a RDS file. To run the code below, you need to set the stock_token environment variable to your API token. In our case, we saved the token as an environment variable using the usethis::edit_r_environ(scope = 'project') function.

# pull the stock_api_token from the environment variable
stock_api_token = Sys.getenv("stock_token")

# crafting the request for the API
request = paste0(
  "https://stocknewsapi.com/api/v1/category?", # The base URL for the API
  "section=alltickers", # We are interested in all tickers
  "&sentiment=positive,negative", # We want non neutral sentiments
  "&type=article", # We are interested in articles
  "&items=100", # We want 100 items per page
  "&date=01052025-03132025", # The date range
  "&exchange=NYSE,NASDAQ", # We are interested in the NYSE and NASDAQ
  "&country=USA", # We are interested in the USA
  "&token=", stock_api_token, # The API token (ours is saved as an ENV Variable)
  "&page=") # the pages that we will iterate over

# crafting all requests
all_requests = paste0(request, 1:100)

# pulling the data from the API and cleaning the data prior to saving it
stock_news_df = purrr::map_df(all_requests, ~jsonlite::fromJSON(.x)$data) |> 
  # convert the topics and tickers cols into chr (they are lists of strings)
  dplyr::mutate(tickers = purrr::map_chr(tickers, ~ paste(.x, collapse = ", "))) |> 
  dplyr::mutate(topics = purrr::map_chr(topics, ~ paste(.x, collapse = ", "))) |> 
  # convert the date column from character to datetime
  dplyr::mutate(date = lubridate::dmy_hms(date, tz = "America/New_York"))

# writing the data frame as RDS and CSV files
readr::write_rds(stock_news_df, "../data/stock_news_data.rds")
readr::write_csv(stock_news_df, "../data/stock_news_data.csv")

The CSV file containing the stock news data can be accessed here. The stock_news_df data frame contains 10,000 rows and 10 columns. The names of the columns are: news_url, image_url, title, text, source_name, date, topics, sentiment, type, and tickers. Furthermore, the sentiment of the articles is stored in the sentiment column. The sentiment of those articles is divided into two categories: positive, and negative. Their distribution is as follows:

Negative	Positive
2912	7088

4.2 Downsampling the Binary Classification Dataset

In this subsection, we filtered out rows with multiple tickers in the tickers column, keeping only rows with a single ticker (to ensure data quality). Additionally, we ensured the uniqueness of tickers during down sampling by selecting distinct tickers.

set.seed(2025) # set the seed for reproducibility

binary_data = stock_news_df |>
  dplyr::filter(!stringr::str_detect(tickers, ",")) |> # keep rows without commas (i.e., single tickers)
  dplyr::group_by(sentiment) |> # group by sentiment
  dplyr::distinct(tickers, .keep_all = TRUE) |>  # keep unique tickers
  # randomly select <=675 samples per sentiment class (will select all if n < 675)
  dplyr::slice_sample(n = 675) |> 
  dplyr::ungroup() # ungroup the data frame

# save the downsampled dataset as RDS and CSV files
readr::write_rds(binary_data, "../data/binary_classification_data.rds")
readr::write_csv(binary_data, "../data/binary_classification_data.csv")

The resulting binary classification dataset

contains 1,350 samples. The distribution of the sentiment classes is as follows:

Negative	Positive
675	675

4.3 LLM-based Labeling

4.3.1 Prompt Construction

For our labeling task, we defined a system prompt that combines Chain of Thought learning with few-shot learning. The system prompt provides instructions to the LLMs on how to categorize the impact of a news article on a stock’s next-day return as either “Positive” or “Negative”. The system prompt also includes simulated examples (involving Bitcoin which is not in our stock news dataset) to guide the LLMs in their classification task. Then, we defined a user prompt that presents the news article’s title, full text, and the stock ticker symbol to the LLMs (these will be obtained from the CSV file). Both prompts are combined into a chat prompt template that will be used to interact with the LLMs.

# define the system prompt for the categorization task
bin_system_prompt = """
Task: You will be provided with a news article's title, full text, and a stock ticker symbol. Categorize the impact of the news article on a stock's next-day return as either "Positive" or "Negative". 

Instructions:
1. Read the title and full text of the article.
2. Analyze how the information might affect the company associated with the given ticker.
3. Identify key factors such as financial performance, market trends, announcements, and industry developments that may influence investor sentiment.
4. Assess the overall tone and its potential impact on the stock price.

Classification Guidelines:
- "Positive": News likely to increase the stock price.
- "Negative": News likely to decrease the stock price.
- Focus on the immediate impact (next day return).
- Weigh the importance of positive vs. negative factors if the article is mixed.

Output Format:
- <analysis>: [Detailed analysis of the article’s impact]
- <classification>: [Final classification: "Positive" or "Negative"]

Note: Your classification must strictly be "Positive" or "Negative" based on the immediate expected impact.

Examples:

Example 1:
Title: Bitcoin Surges as Major Financial Institution Announces BTC Adoption
Text: In a groundbreaking move, a major financial institution announced that it would start offering Bitcoin as part of its investment portfolios. This decision is expected to significantly increase institutional demand for BTC, boosting investor confidence.
Ticker: BTC
<analysis>: The article highlights a major financial institution adopting Bitcoin, which is likely to enhance institutional investment and demand. This news positively affects investor sentiment and suggests an immediate positive impact on BTC's price.
<classification>: Positive

Example 2:
Title: Bitcoin Faces Increased Regulatory Scrutiny Amid Fraud Concerns
Text: Reports have emerged that several governments are planning to implement stricter regulations on cryptocurrency trading, citing concerns about fraud and market manipulation. The regulatory discussions have sparked debate among investors regarding the future of Bitcoin in heavily regulated markets.
Ticker: BTC
<analysis>: The article discusses potential regulatory actions that could negatively influence market sentiment by raising fears of restricted trading and heightened scrutiny. This news suggests a likely negative impact on BTC's price in the immediate term.
<classification>: Negative
"""

# define the user input string template
bin_user_prompt = """
Here is the news article:

<title>
{title}
</title>

<text>
{text}
</text>

The stock ticker symbol you need to consider is: {tickers}
"""

# create the chat prompt template
bin_chat_prompt = ChatPromptTemplate.from_messages([
    ("system", bin_system_prompt),
    ("human", bin_user_prompt),
])

4.3.2 Labeling the Binary Classification Dataset

In this section, we will use the LLMs to label the sentiment of the news articles in our binary classification data set. We will use our generalized_chat_completion function to interact with the LLMs and generate chat completions for each news article. The chat completions will include the LLM’s analysis and classification of the news article’s impact on the stock’s next-day return. We will run the labeling process for each LLM model and replicate the process three times to ensure consistency in the labeling results.

res = generalized_chat_completion(
  csv_path = '../data/binary_classification_data.csv',
  columns_to_keep = ['date', 'title', 'text', 'tickers'],
  models = models,
  chat_prompt_template = bin_chat_prompt,
  columns_for_chat_prompt = ['title', 'text', 'tickers'],
  num_replicates = 5,
  output_file = '../results/binary_classification_results.csv'
  )

The labeling process has been completed for the binary classification data set. The results have been saved to a CSV file, which can be accessed here.

4.3.3 Task Completion Time by Model

We present a detailed summary of inference latency performance across models below.

# * Summary Table ---------------------------------------------------------

results = readr::read_csv("../results/binary_classification_results.csv")

results |>  
  dplyr::filter(chat_model != 'exaone-deep:2.4b') |> 
  dplyr::mutate(
    chat_model = forcats::fct_relevel(
      chat_model,
      "claude-3-7-sonnet-20250219",
      "claude-3-5-haiku-20241022",
      "gpt-4o-2024-11-20",
      "gpt-4o-mini-2024-07-18",
      "command-r-plus-08-2024",
      "command-r7b",
      "deepseek-r1:7B",
      "deepseek-r1:1.5B",
      "gemma3:27B",
      "gemma3:1B",
      "llama3.2:3B", 
      "llama3.2:1B",
      'phi4:latest',
      'phi4-mini'
    )
  ) |> 
  dplyr::group_by(chat_model) |> 
  dplyr::mutate(
    time_diff = chat_date - dplyr::lag(chat_date)
  ) |> dplyr::select(chat_model, chat_date, time_diff) |> 
  dplyr::summarise(
    n = dplyr::n(),
    min_t = min(time_diff, na.rm = TRUE),
    q25_t = quantile(time_diff, 0.25, na.rm = TRUE),
    mean_t = mean(time_diff, na.rm = TRUE),
    med_t = median(time_diff, na.rm = TRUE),
    q75_t = quantile(time_diff, 0.75, na.rm = TRUE),
    p95_t = quantile(time_diff, 0.95, na.rm = TRUE),
    max_t = max(time_diff, na.rm = TRUE),
  ) |> 
  dplyr::mutate( 
    color = c(rep(c('exp', 'cheap'), 7) ),
    company = c('Anthropic', 'Anthropic', 'Cohere', 'Cohere', 'DeepSeek', 'DeepSeek',
                'Google', 'Google', 'OpenAI', 'OpenAI', 'Meta', 'Meta', 'Microsoft', 'Microsoft')
    ) -> summary_table

# * Plotting the box plot -------------------------------------------------

# create "fake" spacer levels for spacing the boxplot:
model_levels = c( "spacer-0",
  "claude-3-7-sonnet", "claude-3-5-haiku",  "spacer-2",
  "gpt-4o",            "gpt-4o-mini",       "spacer-10a", "spacer-10b",
  "command-r-plus",    "command-r7b",       "spacer-4",
  "deepseek-r1:7B",    "deepseek-r1:1.5B",  "spacer-6",
  "gemma3:27B",        "gemma3:1B",         "spacer-8",
  "llama3.2:3B",       "llama3.2:1B",       "spacer-12",
  "phi4:latest",       "phi4-mini",         "spacer-14"
)

# boxplot data:
results |>  
  dplyr::left_join(summary_table, by = "chat_model") |>
  dplyr::mutate(
    time_diff = (chat_date - dplyr::lag(chat_date)) |> 
      stringr::str_remove_all(" secs") |> as.numeric(),
    chat_model = forcats::fct_recode(
      chat_model,
      `claude-3-7-sonnet` = "claude-3-7-sonnet-20250219",
      `claude-3-5-haiku` = "claude-3-5-haiku-20241022",
      `command-r-plus` = "command-r-plus-08-2024",
      `gpt-4o` = "gpt-4o-2024-11-20",
      `gpt-4o-mini` = "gpt-4o-mini-2024-07-18"
    ) |> 
      factor(levels = model_levels)
  ) |> 
  na.omit() -> boxplot_df

# annotation df:
label_df = boxplot_df |>
  dplyr::group_by(chat_model) |>
  dplyr::summarise(
    min = round(quantile(time_diff, 0)),
    med = round(quantile(time_diff, 0.5)),
    max = round(quantile(time_diff, 1)),
    color = dplyr::first(color),
    .groups = "drop"
  ) |>
  tidyr::pivot_longer(
    cols = c(min, med, max),
    names_to = "stat",
    values_to = "label"
  ) |>
  dplyr::arrange(factor(chat_model, levels = model_levels)) |>
  dplyr::mutate(
    group_index = rep(seq_len(length(unique(chat_model))), each = 3),
    base_nudge = ifelse(group_index %% 2 == 1, -1, 1),
    # Apply tighter nudge for min/max, looser for median
    y_nudge = dplyr::case_when(
      stat == "med" ~ base_nudge * 0.725,
      stat %in% c("min", "max") ~ base_nudge * 0.5
    )
  ) |> 
  dplyr::filter( !(chat_model == "llama3.2:1B" & stat == 'min') )


# data for the rectangle boxes:
proprietary_models = c("claude-3-7-sonnet", "claude-3-5-haiku", "gpt-4o", "gpt-4o-mini")
y_indices = which(model_levels %in% proprietary_models)
y_min = min(y_indices) - 0.5
y_max = max(y_indices) + 0.5

open_api = c("command-r-plus")
y_indices2 = which(model_levels %in% open_api)
y_min2 = min(y_indices2) - 0.5
y_max2 = max(y_indices2) + 0.5

ollama_models = c(
  'command-r7b', 'deepseek-r1:7B', 'deepseek-r1:1.5B', 'gemma3:27B', 'gemma3:1B',
  'llama3.2:3B', 'llama3.2:1B', 'phi4:latest', 'phi4-mini'
)
y_indices3 = which(model_levels %in% ollama_models)
y_min3 = min(y_indices3) - 0.5
y_max3 = max(y_indices3) + 0.5


# colors for ylabels
color_lookup = boxplot_df |>
  dplyr::distinct(chat_model, color) |>
  dplyr::filter(!grepl("^spacer", chat_model)) |>
  dplyr::mutate(
    label_colored = dplyr::case_when(
      color == "exp" ~ paste0("<span style='color:#C3142D'>", chat_model, "</span>"),
      color == "cheap" ~ paste0("<span style='color:#808080'>", chat_model, "</span>"),
      TRUE ~ chat_model
    )
  )

model_labels = setNames(color_lookup$label_colored, color_lookup$chat_model)

# Add empty labels for spacers
spacer_labels = rep("", sum(grepl("^spacer", model_levels)))
names(spacer_labels) = model_levels[grepl("^spacer", model_levels)]

# Merge
full_labels = c(model_labels, spacer_labels)
full_labels = full_labels[model_levels]  # Reorder to match axis


# plot:
boxplot_df |>
  dplyr::mutate(chat_model = factor(chat_model, levels = model_levels)) |> 
  ggplot2::ggplot(ggplot2::aes(y = chat_model, x = time_diff)) +
  ggplot2::geom_boxplot(ggplot2::aes(color = color), size =0.5) +  
  
  # adding blanks between pairs of models
  ggplot2::geom_blank(data = data.frame(chat_model = model_levels, time_diff = NA)) + 
  
  # use ggplot2::stat_summary to annotate the max for each model
  ggplot2::geom_text(
    data = label_df,
    ggplot2::aes(x = label, y = chat_model, label = label, color = color),
    position = ggplot2::position_nudge(y = label_df$y_nudge),
    size = 2.75,
  ) +
  
  ggplot2::scale_x_log10() +
  ggplot2::scale_y_discrete(
    limits = model_levels,
    labels = full_labels
  ) +
  ggplot2::scale_color_manual(values = c("exp" = "#C3142D", "cheap" = "#808080")) +
  
  ggplot2::annotate("rect",
             xmin = 0.9, xmax = 1200,
             ymin = y_min-0.8, ymax = y_max+0.6,
             alpha = 0.02,
             fill = NA,
             color = "black",
             linetype = 3
  ) +
  ggplot2::annotate("text",
           x = 310,
           y = y_max + .35,  # just inside the box
           label = "Proprietary LLMs\n (via API)",
           hjust = 0.5,
           vjust = 1,
           fontface = "bold",
           size = 3
  ) +
  ggplot2::annotate("rect",
                    xmin = 0.9, xmax = 1200,
                    ymin = y_min3-1.6, ymax = y_max3+0.6,
                    alpha = 0.02,
                    color = "black",
                    fill = NA,
                    linetype = 2
  ) +
  ggplot2::annotate("text",
                    x = 120,
                    y = y_max3+0.35 ,  # just inside the box
                    label = "Open Weight LLMs via Ollama\n (command-r-plus via API)",
                    hjust = 0.5,
                    vjust = 1,
                    fontface = "bold",
                    size = 3
  ) +
  ggplot2::labs(
    title = "Task Completion Time by Model",
    subtitle = "<span style='color:#808080'>Cheaper</span> vs. <span style='color:#C3142D'>more expensive (time, cost) </span> LLMs by company",
    y = "",
    x = "Task Completion Time in Seconds (Log Scale)",
    caption = 'Annotated with min, median, and max times based on n=6,750 annotations per model.'
       ) +
  ggplot2::theme_classic() +
  ggplot2::theme(
    plot.title = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 11),
    plot.subtitle = ggtext::element_markdown(hjust = 1, face = "bold", size = 10),
    axis.title.x = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
    axis.title.y = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
    axis.text.x = ggtext::element_markdown(hjust = 0.5, size = 9, face = 'bold'),
    axis.text.y = ggtext::element_markdown(hjust = 1, size = 9, face = 'bold'),
    panel.grid.major = ggplot2::element_blank(),
    panel.grid.minor = ggplot2::element_blank(),
    legend.position = "none",
    plot.caption = ggtext::element_markdown(hjust = 0.95, size = 8),
    axis.ticks.y = ggplot2::element_blank()
  )

4.3.4 Extracting the LLM Labels and Creating a Data Frame for Analysis

In this section, we will extract the LLM labels from the binary classification results and create a data frame for further analysis. We will extract the LLM labels from the chat responses and create a new column in the data frame for each LLM model. We will also calculate the frequency of the most common label across replicates to assess the consistency of the LLMs.

pacman::p_load(irrCAC, stringr, tidyverse)

# helper function to calculate the frequency of the most common value
mode_frequency = function(x, drop_na=F) {
  # return 0 if all values are NA
  if (all(is.na(x))) {
    return(0)
  }
  # calculate the frequency table
  freq_table = table(x)
  if (drop_na==F){
    # return the frequency of the most common value
    return(max(freq_table) / length(x))
  }else{
    return(max(freq_table)/sum(freq_table))
  }
}

# read the binary classification results from the CSV file
binary_classification_results = readr::read_csv("../results/binary_classification_results.csv")
binary_classification_results = binary_classification_results |>
  filter(!(chat_model %in% c("exaone-deep:2.4b")))

# allowable words for the classification
valid_words <- c("Negative", "Positive", "Neutral", "Mixed", "Inconclusive", "Insufficient", "Impossible", "Ambiguous", "Indeterminate")

# extract the LLM labels from the chat responses
binary_classification_results = binary_classification_results |>
  mutate(llm_label = sapply(chat_response, extract_classification, valid_words = valid_words),
         llm_label = unname(llm_label),
         llm_label = str_to_title(llm_label),
         llm_label = ifelse(llm_label %in% c("Neutral", "Mixed", "Inconclusive", "Insufficient", "Impossible", "Ambiguous", "Indeterminate"), NA, llm_label),
         llm_label = as.factor(llm_label)
         ) |>
  # select relevant columns
  dplyr::select(date, title, text, tickers, chat_model, chat_replicate, llm_label)

# print the frequency of llm_label
print(table(binary_classification_results$llm_label, useNA = "always"))

# pivot the data to wide format for easier comparison (and append the replicate number in name)
binary_analysis_df <- binary_classification_results |> 
  # pivot the data to wide format for easier comparison (and append the replicate number in name)
  tidyr::pivot_wider(names_from = chat_replicate, values_from = llm_label, names_prefix = "rep_") |>
  # compute the frequency of the most frequent value across replicates
  dplyr::rowwise() |>
  dplyr::mutate(
    percent_agreement = mode_frequency(
      dplyr::c_across(tidyselect::starts_with("rep_"))
    ) * 100,
    percent_agreement_drop_na = mode_frequency(
      dplyr::c_across(tidyselect::starts_with("rep_")), drop_na=T
    ) * 100
  ) |>
  dplyr::ungroup() # ungroup after rowwise operation

# save the analysis dataframe as RDS and CSV files
readr::write_rds(binary_analysis_df, "../results/binary_analysis_df.rds")
readr::write_csv(binary_analysis_df, "../results/binary_analysis_df.csv")

The analysis dataframe for the binary classification experiment has been created. The dataframe contains the LLM labels for each model and replicate, along with the frequency of the most common label across replicates. The results can be accessed in the binary_analysis_df CSV file.

4.3.5 Distribution of LLMs for Binary Classification

In this section, we study the distribution of LLM labels for binary classification. The figure below, the “invalid” label represents the inconsistency between our prompt (which explicitly instructed the LLM to restrict its classification to either “positive” or “negative”) and the generated classification, which sometimes included terms such as “neutral” or “unsure”.

# library(tidyverse)

# * Getting the Data ------------------------------------------------------
miamired = '#C3142D'
binary_classification_results = 
  readr::read_csv("../results/binary_analysis_df.csv") |> 
  dplyr::select(
    date, title, tickers:rep_5
  ) |> 
  tidyr::pivot_longer(
    cols = rep_1:rep_5,
    names_to = "rep",
    values_to = "llm_label"
  ) |>  
  dplyr::filter(chat_model != 'exaone-deep:2.4b') 


# * Stuff Needed for Coloring ---------------------------------------------

# * Chat Models -----------------------------------------------------------

# Assign alternating colors to chat_model labels
chat_models = c(
  "phi4-mini",
  "phi4:latest",
  "llama3.2:1B",
  "llama3.2:3B", 
  "gemma3:1B",
  "gemma3:27B",
  "deepseek-r1:1.5B",
  "deepseek-r1:7B",
  "command-r7b",
  "command-r-plus-08-2024",
  "gpt-4o-mini-2024-07-18",
  "gpt-4o-2024-11-20",
  "claude-3-5-haiku-20241022",
  "claude-3-7-sonnet-20250219"
)

model_abbrev = c(
  "phi4-mini"                 = "phi4-mini",
  "phi4:latest"               = "phi4:latest",
  "llama3.2:1B"               = "llama3.2:1B",
  "llama3.2:3B"               = "llama3.2:3B",
  "gemma3:1B"                 = "gemma3:1B",
  "gemma3:27B"                = "gemma3:27B",
  "deepseek-r1:1.5B"          = "deepseek-r1:1.5B",
  "deepseek-r1:7B"            = "deepseek-r1:7B",
  "command-r7b"               = "command-r7b",
  "command-r-plus-08-2024"    = "command-r-plus",
  "gpt-4o-mini-2024-07-18"    = "gpt-4o-mini",
  "gpt-4o-2024-11-20"         = "gpt-4o",
  "claude-3-5-haiku-20241022" = "claude-3-5-haiku",
  "claude-3-7-sonnet-20250219"= "claude-3-7-sonnet"
)

# Apply alternating color span tags
label_colors = c("Negative" = "#1F78B4", "Invalid" = "#B0B0B0", "Positive" = "#A6CEE3")


# * Plotting ---------------------------------------------------------------
plots = purrr::imap(chat_models, function(model, idx) {
  color = if (idx %% 2 == 1) "#808080"  else miamired
  model_short = model_abbrev[[model]]
  
  binary_classification_results |>
    dplyr::filter(chat_model == model) |>
    dplyr::mutate(
      llm_label = forcats::fct_expand(llm_label, "Invalid"),
      llm_label = forcats::fct_na_value_to_level(llm_label, "Invalid"),
      llm_label = forcats::fct_relevel(llm_label, "Positive", "Invalid", "Negative")
    ) |>
    dplyr::group_by(llm_label) |>
    dplyr::summarize(count = dplyr::n()) |>
    dplyr::mutate(percent = count / sum(count) * 100) |>
    ggplot2::ggplot( ggplot2::aes(x = llm_label, y = percent, fill = llm_label)) +
    ggplot2::geom_bar(stat = "identity", position = "dodge") +
    ggplot2::geom_text( ggplot2::aes(label = sprintf("%.2f%%", percent)), 
                        vjust = -0.5, size = 3, fontface = "bold") +
    ggplot2::scale_fill_manual(values = label_colors) +
    ggplot2::scale_y_continuous(
      limits = c(0, 110),
      breaks = seq(0, 100, 50),
      labels = scales::label_percent(scale = 1)
    ) +
    ggplot2::ggtitle(model_short) +
    ggplot2::theme_classic() +
    ggplot2::theme(
      plot.title =  ggplot2::element_text(color = color, hjust = 0.5, face = "bold", size = 8),
      axis.title.x =  ggplot2::element_blank(),
      axis.title.y =  ggplot2::element_blank(),
      axis.text =  ggplot2::element_text(size = 8, face = 'bold'),
      legend.position = "none"
    )
})

# Combine plots using patchwork
patchwork::wrap_plots(plots, ncol = 2, axis_titles ='collect') +
  patchwork::plot_annotation(
    title = "Distribution of LLM Labels for Binary Classification",
    subtitle = "<span style='color:#808080'>Cheaper</span> vs. <span style='color:#C3142D'>more expensive (time, cost) </span> LLMs by company",
    theme = ggplot2::theme(
      plot.title = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 11),
      plot.subtitle = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 10),
      axis.title.x = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
      axis.title.y = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
      axis.text.x = ggtext::element_markdown(hjust = 0.5, size = 7),
      axis.text.y = ggtext::element_markdown(hjust = 1, size = 7),
      strip.text = ggtext::element_markdown(face = "bold", size = 8),
      panel.grid.major = ggplot2::element_blank(),
      panel.grid.minor = ggplot2::element_blank(),
      legend.position = "none",
    )
  )

4.4 Reliability Assessment

In this section, we conduct the reliability assessment.

4.4.1 Distribution of NA-Penalized Agreement Across Articles by LLM

The figure below shows the NA-penalized agreement distributions for each model, highlighting the percentage of articles where all five replicates generated identical classifications. The NA-penalty reflects how we count agreement in the presence of invalid responses, where NAs are ignored in the numerator but are kept in the denominator.

# library(tidyverse)
miamired = '#C3142D'

chat_model_colors = rep(c('#808080', '#C3142D'), 7)

mode_frequency = function(x, drop_na=F) {
  # return 0 if all values are NA
  if (all(is.na(x))) {
    return(0)
  }
  # calculate the frequency table
  freq_table = table(x)
  if (drop_na==F){
    # return the frequency of the most common value
    return(max(freq_table) / length(x))
  }else{
    return(max(freq_table)/sum(freq_table))
  }
}


# * Getting the Data ------------------------------------------------------

binary_classification_results = 
  readr::read_csv("../results/binary_analysis_df.csv") |> 
  dplyr::select(
    date, title, tickers:rep_5
  ) |> 
  tidyr::pivot_longer(
    cols = rep_1:rep_5,
    names_to = "rep",
    values_to = "llm_label"
  ) |>  
  dplyr::filter(chat_model != 'exaone-deep:2.4b') 



# * Stuff Needed for Coloring ---------------------------------------------

# * Chat Models -----------------------------------------------------------

# Assign alternating colors to chat_model labels
chat_models = c(
  "phi4-mini",
  "phi4:latest",
  "llama3.2:1B",
  "llama3.2:3B", 
  "gemma3:1B",
  "gemma3:27B",
  "deepseek-r1:1.5B",
  "deepseek-r1:7B",
  "command-r7b",
  "command-r-plus-08-2024",
  "gpt-4o-mini-2024-07-18",
  "gpt-4o-2024-11-20",
  "claude-3-5-haiku-20241022",
  "claude-3-7-sonnet-20250219"
)

model_abbrev = c(
  "phi4-mini"                = "phi4-mini",
  "phi4:latest"              = "phi4:latest",
  "llama3.2:1B"               = "llama3.2:1B",
  "llama3.2:3B"               = "llama3.2:3B",
  "gemma3:1B"                 = "gemma3:1B",
  "gemma3:27B"                = "gemma3:27B",
  "deepseek-r1:1.5B"          = "deepseek-r1:1.5B",
  "deepseek-r1:7B"            = "deepseek-r1:7B",
  "command-r7b"               = "command-r7b",
  "command-r-plus-08-2024"    = "command-r-plus",
  "gpt-4o-mini-2024-07-18"    = "gpt-4o-mini",
  "gpt-4o-2024-11-20"         = "gpt-4o",
  "claude-3-5-haiku-20241022" = "claude-3-5-haiku",
  "claude-3-7-sonnet-20250219"= "claude-3-7-sonnet"
)


# * Per Task and Model Labels ---------------------------------------------

binary_classification_results |>  
  tidyr::pivot_wider(
    names_from = rep,
    values_from = llm_label
  ) |> 
  dplyr::select(-date) -> binary_classification_results_wide

binary_analysis_df = 
  binary_classification_results_wide |>
  dplyr::rowwise() |>
  dplyr::mutate(
    percent_agreement = 100* mode_frequency(dplyr::c_across(starts_with("rep_")))
  ) |> 
  dplyr::ungroup()

agreement = 
  binary_analysis_df |> 
  dplyr::group_by(chat_model, percent_agreement) |> 
  dplyr::summarize(count = dplyr::n()) |>
  dplyr::ungroup() |> 
  dplyr::group_by(chat_model) |>
  dplyr::mutate(
    percent = 100 * (count / sum(count)),
    chat_model = model_abbrev[chat_model] |> as.factor(),
    chat_model = forcats::fct_relevel(
      chat_model, 
      "phi4-mini", "phi4:latest", "llama3.2:1B", "llama3.2:3B", 
      "gemma3:1B", "gemma3:27B", "deepseek-r1:1.5B", "deepseek-r1:7B", 
      "command-r7b", "command-r-plus", "gpt-4o-mini", "gpt-4o", 
      "claude-3-5-haiku", "claude-3-7-sonnet"
    )
  ) |>
  dplyr::ungroup()


# * Plot List -------------------------------------------------------------

chat_model_colors = setNames(chat_model_colors, unique(agreement$chat_model) )

# Split data by chat_model and get the names for titles
chat_model_groups = agreement |> 
  dplyr::group_by(chat_model) |> 
  dplyr::group_split()

chat_model_names = agreement |> 
  dplyr::group_by(chat_model) |> 
  dplyr::group_keys() |> 
  dplyr::pull(chat_model)

# Create a list of plots with the chat_model as the title
plot_list = purrr::map2(chat_model_groups, chat_model_names, ~{
  title_color = chat_model_colors[[.y]]
  title_html = glue::glue("<span style='color:{title_color}'>{.y}</span>")
  
  ggplot2::ggplot(.x, ggplot2::aes(x = percent_agreement, y = percent, fill = chat_model)) +
    ggplot2::geom_bar(stat = 'identity', color = 'black') +
    ggplot2::scale_fill_manual(values = chat_model_colors) +
    ggplot2::labs(
      title = title_html,
      x = "Percent Agreement within a LLM (NA Penalty)",
      #y = 'Percentage of Articles',
      y = '',
      fill = "LLM Model"
    ) +
    ggplot2::scale_x_continuous(breaks = seq(0, 100, 20)) +
    ggplot2::scale_y_continuous(
      labels = scales::percent_format(scale = 1),
      limits = c(0, 160),
      breaks = seq(0, 100, 50)
    ) +
    ggplot2::geom_text(
      ggplot2::aes(label = sprintf("%.1f%%", percent)),
      vjust = -0.5,
      size = 2.5,
      fontface = "bold"
    ) +
    ggplot2::theme_classic() +
    ggplot2::theme(
      plot.title = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
      axis.title.x = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
      axis.title.y = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
      axis.text.x = ggtext::element_markdown(hjust = 0.5, size = 7, face = "bold"),
      axis.text.y = ggtext::element_markdown(hjust = 1, size = 7, face = "bold"),
      strip.text = ggtext::element_markdown(face = "bold", size = 8),
      panel.grid.major = ggplot2::element_blank(),
      panel.grid.minor = ggplot2::element_blank(),
      legend.position = "none"
    )
})


patchwork::wrap_plots(plot_list, ncol = 2, axis_titles ='collect', axes = 'keep') +
  patchwork::plot_annotation(
    title = "Distribution of NA-Penalized Agreement Across Articles by LLM",
    subtitle = "<span style='color:#808080'>Cheaper</span> vs. <span style='color:#C3142D'>more expensive (time, cost) </span> LLMs by company",
    theme = ggplot2::theme(
      plot.title = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
      plot.subtitle = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
      axis.title.x = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
      axis.title.y = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
      axis.text.x = ggtext::element_markdown(hjust = 0.5, size = 7),
      axis.text.y = ggtext::element_markdown(hjust = 1, size = 7),
      strip.text = ggtext::element_markdown(face = "bold", size = 8),
      panel.grid.major = ggplot2::element_blank(),
      panel.grid.minor = ggplot2::element_blank(),
      legend.position = "none",
    )
  ) -> combined_plot

cowplot::ggdraw() +
  cowplot::draw_label(
    "Percentage of Articles",
    x = 0.02, y = 0.5, angle = 90, vjust = 0.5,
    fontface = "bold", size = 9
  ) +
  cowplot::draw_plot(combined_plot, x = 0.05, y = 0, width = 0.95, height = 1)

4.4.2 Computing the Reliability Metrics

pacman::p_load(irrCAC, stringr, tidyverse)
binary_analysis_df <- readr::read_csv("../results/binary_analysis_df.csv")

# binary classification reliability metrics
df = reliability_coefs(binary_analysis_df, 6:10)
df = df |>
  mutate(across(3:6, ~round(.x,3)))

cat("The reliability metrics for the binary classification experiment are as follows:\n")

The reliability metrics for the binary classification experiment are as follows:

DT::datatable(df, rownames = FALSE, options = list(pageLength = 10, scrollX = TRUE))

cat("The mean percent agreement with NA penalty for each model is as follows:\n")

The mean percent agreement with NA penalty for each model is as follows:

pa_table = pa_summary(binary_analysis_df, "percent_agreement", digits = 3) |> 
  tidyr::separate(
    mPa, into = c("mean_percent_agreement", "sd_percent_agreement"), 
    sep = " \\("
  ) |>
  dplyr::mutate(
    mean_percent_agreement = as.numeric(mean_percent_agreement),
    sd_percent_agreement = 
      as.numeric(stringr::str_remove(sd_percent_agreement, "\\)"))
  )

DT::datatable(pa_table, rownames = FALSE, 
              options = list(pageLength = 14, scrollX = TRUE)
)

cat(
  paste(
    'From the table above, we can see that the standard deviation of the percent agreement is quite low for,',
    paste(pa_table$model[which(pa_table$sd_percent_agreement < 0.1)], collapse = ', '),
    'indicating a high level of agreement among those LLMs in the binary classification experiment. However, the others models have higher standard deviations, especially for the',
    paste(pa_table$model[which(pa_table$sd_percent_agreement > 0.15)], collapse = ', '),
    'model. \n')
)

From the table above, we can see that the standard deviation of the percent agreement is quite low for, claude-3-7-sonnet-20250219, command-r-plus-08-2024, gemma3:27B, gpt-4o-2024-11-20, gpt-4o-mini-2024-07-18, llama3.2:1B, phi4:latest indicating a high level of agreement among those LLMs in the binary classification experiment. However, the others models have higher standard deviations, especially for the deepseek-r1:1.5B, deepseek-r1:7B, llama3.2:3B, phi4-mini model.

pa_table_drop_na = pa_summary(binary_analysis_df, "percent_agreement_drop_na", digits = 3) |> 
  tidyr::separate(
    mPa, into = c("mean_percent_agreement_drop_na", "sd_percent_agreement"), 
    sep = " \\("
  ) |>
  dplyr::mutate(
    mean_percent_agreement_drop_na = as.numeric(mean_percent_agreement_drop_na),
    sd_percent_agreement = 
      as.numeric(stringr::str_remove(sd_percent_agreement, "\\)"))
  )

df_wide = df |> 
  dplyr::select(model, coeff.name, coeff.val) |> 
  tidyr::pivot_wider(names_from = coeff.name, values_from = coeff.val)

df_wide = dplyr::left_join(
  x = pa_table |> dplyr::select(-sd_percent_agreement), y = df_wide, 
  by = "model"
) 

df_wide = dplyr::left_join(
  x = pa_table_drop_na |> dplyr::select(-sd_percent_agreement), y = df_wide, 
  by = "model"  
) |> 
  dplyr::rename(
    `Percent Agreement (with NA Penalty)` = mean_percent_agreement,
    `Percent Agreement (Ignoring NA)` = mean_percent_agreement_drop_na
  ) |> 
  dplyr::select(
    model, 
    `Percent Agreement (with NA Penalty)`, `Percent Agreement (Ignoring NA)`, 
    dplyr::everything() 
  )

# get the models with high standard deviation
high_pa_sd_models_index = which(pa_table$sd_percent_agreement > 0.15)

# a summarized table of the metrics
df_wide = df_wide |> 
  # rounding numeric columns to 3 decimal places
  dplyr::mutate(
    dplyr::across(dplyr::where(is.numeric), ~ round(., 3)),
    high_pa_sd = ifelse(
      model %in% pa_table$model[pa_table$sd_percent_agreement > 0.15],
      TRUE, FALSE
    ),
    `API Costs / 1M Input Tokens` = c(0.8, 3, 2.5, 0, 0, 0, 0, 0, 2.5, 0.15, 0, 0, 0, 0),
    `API Costs / 1M Output Tokens`= c(4,  15,  10, 0, 0 ,0, 0, 0,  10,  0.6, 0, 0, 0, 0)
  )

# save the reliability metrics dataframe as RDS and CSV files
readr::write_rds(df_wide, "../results/binary_reliability_metrics_means.rds")
readr::write_csv(df_wide, "../results/binary_reliability_metrics_means.csv")
readr::write_rds(pa_table, "../results/binary_pa_summary.rds")
readr::write_csv(pa_table, "../results/binary_pa_summary.csv")
readr::write_rds(df, "../results/binary_reliability_metrics.rds")
readr::write_csv(df, "../results/binary_reliability_metrics.csv")

The CSV files containing the reliability metrics for the binary classification experiment can be accessed at binary_reliability_metrics_means.csv, binary_pa_summary.csv, and binary_reliability_metrics.csv.

4.4.3 Intra-LLM Reliability

In this section, we study the chance-adjusted reliability coefficient estimates (dots) and their id'{a}k-adjusted confidence intervals (whiskers), constructed to maintain 90% family-wise confidence within each cost group.

# library(tidyverse)
library(grid)
library(gridExtra)

# ---- Setup ----
chat_models <- c(
  "phi4-mini",
  "llama3.2:1B",
  "gemma3:1B",                
  "deepseek-r1:1.5B",
  "command-r7b",
  "gpt-4o-mini-2024-07-18",
  "claude-3-5-haiku-20241022",
  "phi4:latest", 
  "llama3.2:3B", 
  "gemma3:27B",
  "deepseek-r1:7B",
  "command-r-plus-08-2024",
  "gpt-4o-2024-11-20",
  "claude-3-7-sonnet-20250219"
)

model_abbrev = c(
  "phi4-mini"                 = "phi4-mini",
  "llama3.2:1B"              = "llama3.2:1B",
  "gemma3:1B"                = "gemma3:1B",
  "deepseek-r1:1.5B"         = "deepseek-r1:1.5B",
  "command-r7b"              = "command-r7b",
  "gpt-4o-mini-2024-07-18"   = "gpt-4o-mini",
  "claude-3-5-haiku-20241022"= "claude-3-5-haiku",
  "phi4:latest"              = "phi4:latest",
  "llama3.2:3B"              = "llama3.2:3B",
  "gemma3:27B"               = "gemma3:27B",
  "deepseek-r1:7B"           = "deepseek-r1:7B",
  "command-r-plus-08-2024"   = "command-r-plus",
  "gpt-4o-2024-11-20"        = "gpt-4o",
  "claude-3-7-sonnet-20250219" = "claude-3-7-sonnet"
)

chat_model_colors = rep(c('#808080', '#C3142D'), each = 7)
chat_model_colors = stats::setNames(chat_model_colors, model_abbrev[chat_models])


model_labels <- purrr::map_chr( model_abbrev[chat_models], ~{
  if (grepl("^spacer", .x)) return("")
  glue::glue("<span style='color:{chat_model_colors[.x]}'><b>{.x}</b></span>")
})
names(model_labels) <- model_abbrev[chat_models]

alpha = (1-(1-.1)^(1/7))


# ---- Load Data ----
intra_df = readr::read_csv("../results/binary_reliability_metrics.csv") |>
  dplyr::mutate(
    lower_ci = coeff.val - stats::qnorm(1-alpha/2) * coeff.se,
    upper_ci = coeff.val + stats::qnorm(1-alpha/2)* coeff.se,
    abbrev = factor(model_abbrev[model]),
    abbrev = forcats::fct_relevel(abbrev, rev(model_abbrev[chat_models]) )
  )

intra_wide = intra_df |> 
  dplyr::select(abbrev, coeff.name, coeff.val) |> 
  tidyr::pivot_wider(names_from = coeff.name, values_from = coeff.val) |> 
  dplyr::arrange(dplyr::desc(abbrev)) 
colnames(intra_wide) <- c("LLM", "C", "F", 'AC', "BP", "K")


intra_wide = intra_wide |> 
  dplyr::select(LLM, AC, BP, C, F, K) |> 
  dplyr::mutate(
    dplyr::across(where(is.numeric), ~ ifelse(is.na(.x), "--", sprintf("%.2f", .x)))
  )


# *Plot the Data ----------------------------------------------------------

intra_df |> 
  ggplot2::ggplot(ggplot2::aes(x = abbrev, y = coeff.val)) +
  ggplot2::geom_blank() +
  ggplot2::geom_point(ggplot2::aes(color = abbrev), size = 1.5, na.rm = TRUE) +
  ggplot2::geom_errorbar(
    ggplot2::aes(ymin = lower_ci, ymax = upper_ci, color = abbrev),
    width = 0.5,
    na.rm = TRUE
  ) +
  # Annotate lower CI values
  ggplot2::geom_text(
    data = intra_df,
    ggplot2::aes(
      y = lower_ci,
      label = sprintf("%.3f", lower_ci),
      color = abbrev
    ),
    size = 2.25,
    hjust = 1.2,
    na.rm = TRUE
  ) +
  
  # Annotate upper CI values
  ggplot2::geom_text(
    data = intra_df,
    ggplot2::aes(
      y = upper_ci,
      label = sprintf("%.3f", upper_ci),
      color = abbrev
    ),
    size = 2.25,
    hjust = -0.2,
    na.rm = TRUE
  ) +
  ggplot2::scale_color_manual(values = chat_model_colors, na.translate = FALSE) +
  ggplot2::scale_x_discrete(labels = model_labels) +
  ggplot2::scale_y_continuous(limits = c(0.785, 1.053), breaks = seq(0.8, 1.0, 0.05)) +
  ggplot2::coord_flip() +
  ggplot2::facet_wrap(~ coeff.name, ncol = 2, scales = "fixed", axes = 'all') +
  ggplot2::labs(
    title = "Intra-LLM Reliability",
    subtitle = "<span style='color:#808080'>Cheaper</span> vs. <span style='color:#C3142D'>more expensive (time, cost) </span> LLMs by company",
    x = NULL,
    y = NULL
  ) +
  ggplot2::theme_minimal() +
  ggplot2::theme(
    plot.title = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
    plot.subtitle = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
    axis.title.x = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
    axis.title.y = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
    axis.text.x = ggtext::element_markdown(hjust = 0.5, size = 7, face = 'bold'),
    axis.text.y = ggtext::element_markdown(hjust = 1, size = 7, face = 'bold'),
    strip.text = ggtext::element_markdown(face = "bold", size = 8),
    legend.position = "none",
    panel.grid = ggplot2::element_blank(),
    axis.line = ggplot2::element_line(color = "black"),
    axis.ticks = ggplot2::element_line(color = 'black')
  ) -> original_plot



# * Adding a table in the empty space at the bottom right -----------------

# Custom theme
booktabs_theme <- ttheme_minimal(
  core = list(
    fg_params = list(fontface = "plain", cex = 0.6, hjust = 0, x = 0.05)
  ),
  colhead = list(
    fg_params = list(fontface = "bold", cex = 0.65, hjust = 0, x = 0.05),
    bg_params = list(fill = NA)  # No shading, like booktabs
  ),
  padding = unit(c(1.7, 1.7), "mm")  # Tighter padding
)

# Base table
table_grob <- gridExtra::tableGrob(intra_wide, rows = NULL, theme = booktabs_theme)
table_grob$heights <- ggplot2::unit(rep(1, nrow(table_grob)), "lines") * 0.5

# Add title row
title_grob <- grid::textGrob(
  "Intra-LLM Coefficient Estimates",
  gp = grid::gpar(fontface = "bold", fontsize = 9),
  x = 0.5, hjust = 0.5
)

# Add title row above the table
table_grob <- gtable::gtable_add_rows(table_grob, heights = unit(1.5, "lines"), pos = 0)
table_grob <- gtable::gtable_add_grob(
  table_grob,
  grobs = title_grob,
  t = 1, l = 1, r = ncol(table_grob)
)

table_width <- sum(table_grob$widths)
table_height <- sum(table_grob$heights)

padding <- ggplot2::unit(.5, "mm")

# Color code the table 
core_llm_indices <- which(
  table_grob$layout$name == "core-fg" & 
    table_grob$layout$l == 1  # column 1
)

model_colors <- rep(c("#808080", "#C3142D"), each=7)  # matches intra_wide

for (i in seq_along(core_llm_indices)) {
  grob_index <- core_llm_indices[i]
  original_gp <- table_grob$grobs[[grob_index]]$gp
  table_grob$grobs[[grob_index]]$gp <- modifyList(original_gp, gpar(col = model_colors[i]))
}

# Add border around entire table using grobTree
bordered_table <- grid::grobTree(
  grid::rectGrob(
    width = table_width+ padding,
    height = table_height + padding,
    gp = grid::gpar(fill = NA, lwd = 0.7, col ='black') 
  ),
  table_grob
)


final_plot <- cowplot::ggdraw() +
  cowplot::draw_plot(original_plot, 0, 0, 1, 1) +  # main plot takes full area
  cowplot::draw_grob(bordered_table, x = 0.742, y = 0.16, width = 0.01, height = 0.01)

final_plot

4.4.4 Inter-Rater Reliability among LLMs

After confirming strong internal consistency within individual models, we examine agreement between different LLMs. Instead of analyzing all possible combinations within each group, we focus on top-N combinations ranked by Krippendorff’s Alpha, with N ranging from 2 to 7.

# ---- Setup ----
chat_models = c(
  "Top 2 Cheaper",
  "Top 2 Expensive",
  "Top 3 Cheaper",
  "Top 3 Expensive",
  "Top 4 Cheaper",
  "Top 4 Expensive",
  "Top 5 Cheaper",
  "Top 5 Expensive",
  "Top 6 Cheaper",
  "Top 6 Expensive",
  "Top 7 Cheaper",
  "Top 7 Expensive"
)

model_abbrev = c(
  "Top 2 Cheaper"   = "Top 2 Cheaper",
  "Top 2 Expensive" = "Top 2 Expensive",
  "Top 3 Cheaper"   = "Top 3 Cheaper",
  "Top 3 Expensive" = "Top 3 Expensive",
  "Top 4 Cheaper"   = "Top 4 Cheaper",
  "Top 4 Expensive" = "Top 4 Expensive",
  "Top 5 Cheaper"   = "Top 5 Cheaper",
  "Top 5 Expensive" = "Top 5 Expensive",
  "Top 6 Cheaper"   = "Top 6 Cheaper",
  "Top 6 Expensive" = "Top 6 Expensive",
  "Top 7 Cheaper"   = "Top 7 Cheaper",
  "Top 7 Expensive" = "Top 7 Expensive"
)

chat_model_abbrev = c(
  "phi4-mini"                 = "phi4-mini",
  "phi4:latest"              = "phi4:latest",
  "llama3.2:1B"              = "llama3.2:1B",
  "llama3.2:3B"              = "llama3.2:3B",
  "gemma3:1B"                = "gemma3:1B",
  "gemma3:27B"               = "gemma3:27B",
  "deepseek-r1:1.5B"         = "deepseek-r1:1.5B",
  "deepseek-r1:7B"           = "deepseek-r1:7B",
  "command-r7b"              = "command-r7b",
  "command-r-plus-08-2024"   = "command-r-plus",
  "gpt-4o-mini-2024-07-18"   = "gpt-4o-mini",
  "gpt-4o-2024-11-20"        = "gpt-4o",
  "claude-3-5-haiku-20241022"= "claude-3-5-haiku",
  "claude-3-7-sonnet-20250219" = "claude-3-7-sonnet"
)

chat_model_colors = rep(c('#808080', '#C3142D'), length.out = length(chat_models))
chat_model_colors = stats::setNames(chat_model_colors, model_abbrev[chat_models])


model_labels <- purrr::map_chr(model_abbrev[chat_models], ~{
  if (grepl("^spacer", .x)) return("")
  glue::glue("<span style='color:{chat_model_colors[.x]}'><b>{.x}</b></span>")
})

names(model_labels) <- model_abbrev[chat_models]


# ---- Load Data ----
intra_df = read_csv("../results/binary_reliability_metrics.csv") %>% 
  filter(coeff.name == "Krippendorff's Alpha") %>% 
  select(model, coeff.val, coeff.se)

intra_df$model <- factor(intra_df$model, levels = c("phi4-mini",                 
  "phi4:latest", "llama3.2:1B", "llama3.2:3B", "gemma3:1B",                
  "gemma3:27B", "deepseek-r1:1.5B", "deepseek-r1:7B",           
  "command-r7b", "command-r-plus-08-2024",
  "gpt-4o-mini-2024-07-18", "gpt-4o-2024-11-20",        
  "claude-3-5-haiku-20241022", "claude-3-7-sonnet-20250219")
  )

intra_df <- intra_df %>% 
  arrange(model) %>% 
  mutate(cost = rep(c("Cheaper", "Expensive"), 7)) %>% 
  select(cost, everything()) %>% 
  arrange(cost, desc(coeff.val))

binary_analysis_df <- read_csv("../results/binary_analysis_df.csv")

df_long <- binary_analysis_df %>%
  pivot_longer(cols = starts_with("rep"), names_to = "rep", values_to = "value")

inter_temp <- df_long %>%
  select(chat_model, value) %>%
  group_by(chat_model) %>%
  mutate(id = row_number()) %>%
  pivot_wider(names_from = chat_model, values_from = value) %>% 
  select(-id)

find_inter_metrics <- function(k, cost){
  models <- as.character(intra_df$model[intra_df$cost==cost])[1:k]
  temp <- inter_temp[models]
  temp$chat_model = paste("Top", k, cost)
  reliability_coefs(temp, 1:k) %>% 
    mutate(lower_ci = coeff.val - stats::qnorm(0.975) * coeff.se,
           upper_ci = coeff.val + stats::qnorm(0.975) * coeff.se) %>% 
    select(model, coeff.name, coeff.val, lower_ci, upper_ci)
}

# Define values to loop over
k_vals <- 2:7
cost_levels <- c("Cheaper", "Expensive")


# Create all combinations of k and cost
combinations <- expand.grid(k = 2:7, cost = c("Cheaper", "Expensive"), KEEP.OUT.ATTRS = FALSE)

# Apply the function for each combination
inter_df <- pmap_dfr(combinations, function(k, cost) {
  find_inter_metrics(k, cost) %>%
    mutate(k = k, cost = cost)
})

inter_df$abbrev <- factor(model_abbrev[inter_df$model])

intra_wide <- intra_df %>% 
  mutate(Rank = rep(paste("Top", 1:7), 2),
         LLM = chat_model_abbrev[model]
  ) %>% 
  select(Rank, LLM) 

inter_wide <- data.frame(Rank = 1:7,
                         Cheaper = intra_wide$LLM[1:7],
                         Expensive = intra_wide$LLM[8:14]) 


row.names(inter_wide) = NULL


# *Plot the Data -------NULL# *Plot the Data ----------------------------------------------------------

inter_df |> 
  ggplot2::ggplot(ggplot2::aes(x = abbrev, y = coeff.val)) +
  ggplot2::geom_blank() +
  ggplot2::geom_point(ggplot2::aes(color = abbrev), size = 1.5, na.rm = TRUE) +
  ggplot2::geom_errorbar(
    ggplot2::aes(ymin = lower_ci, ymax = upper_ci, color = abbrev),
    width = 0.5,
    na.rm = TRUE
  ) +
  # Annotate lower CI values
  ggplot2::geom_text(
    data = inter_df,
    ggplot2::aes(
      y = lower_ci,
      label = sprintf("%.3f", lower_ci),
      color = abbrev
    ),
    size = 2.25,
    hjust = 1.2,
    na.rm = TRUE
  ) +
  
  # Annotate upper CI values
  ggplot2::geom_text(
    data = inter_df,
    ggplot2::aes(
      y = upper_ci,
      label = sprintf("%.3f", upper_ci),
      color = abbrev
    ),
    size = 2.25,
    hjust = -0.2,
    na.rm = TRUE
  ) +
  ggplot2::scale_color_manual(values = chat_model_colors, na.translate = FALSE) +
  ggplot2::scale_x_discrete(labels = model_labels) +
  ggplot2::scale_y_continuous(limits = c(0.615, 0.953), breaks = seq(0.65, 0.9, 0.05)) +
  ggplot2::coord_flip() +
  ggplot2::facet_wrap(~ coeff.name, ncol = 2, scales = "fixed", axes = 'all') +
  ggplot2::labs(
    title = "Inter-LLM Reliability",
    subtitle = "<span style='color:#808080'>Cheaper</span> vs. <span style='color:#C3142D'>more expensive (time, cost) </span> LLMs by company",
    x = NULL,
    y = NULL
  ) +
  ggplot2::theme_minimal() +
  ggplot2::theme(
    plot.title = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
    plot.subtitle = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
    axis.title.x = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
    axis.title.y = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
    axis.text.x = ggtext::element_markdown(hjust = 0.5, size = 7, face = 'bold'),
    axis.text.y = ggtext::element_markdown(hjust = 1, size = 7, face = 'bold'),
    strip.text = ggtext::element_markdown(face = "bold", size = 8),
    legend.position = "none",
    panel.grid = ggplot2::element_blank(),
    axis.line = ggplot2::element_line(color = "black"),
    axis.ticks = ggplot2::element_line(color = 'black')
  ) -> original_plot

# * Adding a table in the empty space at the bottom right -----------------

# Custom theme
booktabs_theme <- ttheme_minimal(
  core = list(
    fg_params = list(fontface = "plain", cex = 0.6, hjust = 0, x = 0.05)
  ),
  colhead = list(
    fg_params = list(fontface = "bold", cex = 0.65, hjust = 0, x = 0.05),
    bg_params = list(fill = NA)  # No shading, like booktabs
  ),
  padding = unit(c(4.5, 4.5), "mm")  # Tighter padding
)

# Base table
table_grob <- gridExtra::tableGrob(inter_wide, rows = NULL, theme = booktabs_theme)
table_grob$heights <- ggplot2::unit(rep(1, nrow(table_grob)), "lines")*0.7


# Add title row
title_grob <- grid::textGrob(
  "LLM Rank based on Kripendorff's Alpha",
  gp = grid::gpar(fontface = "bold", fontsize = 9),
  x = 0.5, hjust = 0.5
)

# Add title row above the table
table_grob <- gtable::gtable_add_rows(table_grob, heights = unit(1.5, "lines"), pos = 0)
table_grob <- gtable::gtable_add_grob(
  table_grob,
  grobs = title_grob,
  t = 1, l = 1, r = ncol(table_grob)
)

table_width <- sum(table_grob$widths)
table_height <- sum(table_grob$heights)

padding <- ggplot2::unit(.5, "mm")

# Color code the table 
core_llm_indices <- which(
  table_grob$layout$name == "core-fg" & 
    table_grob$layout$l == 1  # column 1
)

# Find all core (body cell) grobs
core_indices <- which(table_grob$layout$name == "core-fg")

# Loop over each core cell and color by column and center the first column
for (i in core_indices) {
  col_num <- table_grob$layout[i, "l"]
  original_gp <- table_grob$grobs[[i]]$gp
  grob <- table_grob$grobs[[i]]  # current cell grob
  
  if (col_num == 1) {
    # Center-align column 1 (Rank)
    grob$hjust <- 0.5
    grob$x <- unit(0.5, "npc")
  } else if (col_num == 2) {
    # Cheaper LLM column — gray
    grob$gp <- modifyList(original_gp, gpar(col = "#808080"))
  } else if (col_num == 3) {
    # Expensive LLM column — red
    grob$gp <- modifyList(original_gp, gpar(col = "#C3142D"))
  }
  
  table_grob$grobs[[i]] <- grob
}


# Add border around entire table using grobTree
bordered_table <- grid::grobTree(
  grid::rectGrob(
    width = table_width + padding,
    height = table_height + padding,
    gp = grid::gpar(fill = NA, lwd = 0.7, col ='black') 
  ),
  table_grob
)


final_plot <- cowplot::ggdraw() +
  cowplot::draw_plot(original_plot, 0, 0, 1, 1) +  # main plot takes full area
  cowplot::draw_grob(bordered_table, x = 0.742, y = 0.16, width = 0.01, height = 0.01)

final_plot

4.5 Validity of the Binary Classification Labels (Comparison with “StockNewsAPI” Labels)

In this section, we will compare the LLM labels with the labels provided by the StockNewsAPI (benchmark). We will calculate the agreement between the LLM labels and the “benchmark” labels using various reliability metrics. We will also assess the validity of the LLM labels by comparing them with the “benchmark” labels.

stock_sentiment_df = readr::read_csv("../data/binary_classification_data.csv") |> 
  dplyr::select(date, title, text, tickers, sentiment)

validity_df = 
  readr::read_csv("../results/binary_analysis_df.csv") |> 
  dplyr::select(-percent_agreement) |> 
  dplyr::left_join(stock_sentiment_df, by = c("date", "title", "text", "tickers")) |> 
  # percent of rep_1, rep_2, .., agreement with the benchmark
  dplyr::rowwise() |>
  dplyr::mutate(
    agreement_na_penalty = calculate_agreement(
      reps = dplyr::c_across(dplyr::starts_with("rep_")),
      ground_truth = dplyr::c_across(dplyr::contains("sentiment"))
    ),
    agreement_na_as_na = calculate_agreement(
      reps = dplyr::c_across(dplyr::starts_with("rep_")),
      ground_truth = dplyr::c_across(dplyr::contains("sentiment")),
      na = NA
    )
  )  

# save the validity dataframe as RDS and CSV files
readr::write_rds(validity_df, "../results/binary_sens_validity_df.rds")
readr::write_csv(validity_df, "../results/binary_sens_validity_df.csv")

4.5.1 Visualizing the Validity Metrics

We report accuracy, true positive rate, true negative rate, positive predictive value, and F1 score to provide a comprehensive performance. The figure below shows the LLM classification performance v.s. benchmark (those obtained from the StockNewsAPI). The dot represents each metric’s mean value (from its intra-rater five replicates), and the whiskers length reflect the standard error.

# ---- Setup ----
chat_models = c(
  "phi4-mini",
  "phi4:latest",
  "llama3.2:1B",
  "llama3.2:3B", 
  "gemma3:1B",
  "gemma3:27B",
  "deepseek-r1:1.5B",
  "deepseek-r1:7B",
  "command-r7b",
  "command-r-plus-08-2024",
  "gpt-4o-mini-2024-07-18",
  "gpt-4o-2024-11-20",
  "claude-3-5-haiku-20241022",
  "claude-3-7-sonnet-20250219"
)

model_abbrev = c(
  "phi4-mini"                 = "phi4-mini",
  "phi4:latest"              = "phi4:latest",
  "llama3.2:1B"              = "llama3.2:1B",
  "llama3.2:3B"              = "llama3.2:3B",
  "gemma3:1B"                = "gemma3:1B",
  "gemma3:27B"               = "gemma3:27B",
  "deepseek-r1:1.5B"         = "deepseek-r1:1.5B",
  "deepseek-r1:7B"           = "deepseek-r1:7B",
  "command-r7b"              = "command-r7b",
  "command-r-plus-08-2024"   = "command-r-plus",
  "gpt-4o-mini-2024-07-18"   = "gpt-4o-mini",
  "gpt-4o-2024-11-20"        = "gpt-4o",
  "claude-3-5-haiku-20241022"= "claude-3-5-haiku",
  "claude-3-7-sonnet-20250219" = "claude-3-7-sonnet"
)

chat_model_colors = rep(c('#808080', '#C3142D'), length.out = length(chat_models))
chat_model_colors = stats::setNames(chat_model_colors, model_abbrev[chat_models])


model_labels <- purrr::map_chr( model_abbrev[chat_models], ~{
  if (grepl("^spacer", .x)) return("")
  glue::glue("<span style='color:{chat_model_colors[.x]}'><b>{.x}</b></span>")
})
names(model_labels) <- model_abbrev[chat_models]




# ---- Load Data ----
inter_df = 
  tibble::tribble(
    ~Model,             ~Accuracy, ~Acc_SE, ~TPR,    ~TPR_SE, ~TNR,    ~TNR_SE, ~PPV,    ~PPV_SE, ~F1,    ~F1_SE,
    "phi4-mini",        0.830,     0.001,   0.817,   0.001,   0.843,   0.003,   0.839,   0.003,  0.828,   0.001,
    "phi4:latest",      0.822,     0.001,   0.842,   0.003,   0.802,   0.001,   0.809,   0.000,  0.825,   0.001,
    "llama3.2:1B",      0.864,     0.005,   0.881,   0.003,   0.847,   0.006,   0.852,   0.005,  0.867,   0.004,
    "llama3.2:3B",      0.763,     0.003,   0.797,   0.000,   0.728,   0.005,   0.746,   0.004,  0.770,   0.002,
    "gemma3:1B",        0.876,     0.002,   0.818,   0.008,   0.933,   0.005,   0.924,   0.004,  0.868,   0.003,
    "gemma3:27B",       0.823,     0.001,   0.819,   0.003,   0.828,   0.001,   0.826,   0.001,  0.823,   0.001,
    "deepseek-r1:1.5B", 0.775,     0.008,   0.725,   0.015,   0.825,   0.002,   0.806,   0.005,  0.764,   0.010,
    "deepseek-r1:7B",   0.832,     0.002,   0.835,   0.005,   0.829,   0.001,   0.830,   0.001,  0.832,   0.003,
    "command-r-plus",   0.774,     0.002,   0.715,   0.002,   0.832,   0.006,   0.810,   0.005,  0.760,   0.002,
    "command-r7b",      0.847,     0.003,   0.883,   0.004,   0.810,   0.003,   0.823,   0.003,  0.852,   0.003,
    "gpt-4o-mini",      0.820,     0.003,   0.800,   0.005,   0.840,   0.002,   0.834,   0.003,  0.817,   0.004,
    "gpt-4o",           0.787,     0.007,   0.743,   0.011,   0.832,   0.003,   0.815,   0.004,  0.777,   0.008,
    "claude-3-5-haiku", 0.848,     0.002,   0.882,   0.004,   0.815,   0.003,   0.826,   0.002,  0.853,   0.002,
    "claude-3-7-sonnet",0.831,     0.003,   0.867,   0.004,   0.795,   0.003,   0.809,   0.002,  0.837,   0.003
  )

inter_df = inter_df |> 
  dplyr::rename(Accuracy_mean = Accuracy,
                TPR_mean = TPR,
                TNR_mean = TNR,
                PPV_mean = PPV,
                F1_mean = F1) |>
  dplyr::rename(Accuracy_se = Acc_SE,
                TPR_se = TPR_SE,
                TNR_se = TNR_SE,
                PPV_se = PPV_SE,
                F1_se = F1_SE)


inter_long = inter_df |> 
  tidyr::pivot_longer(
    cols = -Model,
    names_to = c("Metric", ".value"),
    names_sep = "_"
  ) |>
  dplyr::mutate(
    lower = mean - se/sqrt(5),
    upper = mean + se/sqrt(5),
    Model = factor(Model, levels = rev(model_abbrev[chat_models]))
    )

table_df <- inter_df |>
  dplyr::select(Model, Accuracy_mean, TPR_mean, TNR_mean, PPV_mean, F1_mean) %>%
  dplyr::rename(
    LLM = Model,
    ACC = Accuracy_mean,
    TPR = TPR_mean,
    TNR = TNR_mean,
    PPV = PPV_mean,
    F1 = F1_mean) |>
  dplyr::mutate(dplyr::across(dplyr::where(is.numeric), ~ sprintf("%.2f", .))
  )


# *Plot the Data ----------------------------------------------------------

inter_long |> 
  ggplot2::ggplot(ggplot2::aes(x = Model, y = mean, color = Model)) +
  ggplot2::geom_blank() +
  ggplot2::geom_point(size = 0.9, na.rm = TRUE) +
  ggplot2::geom_errorbar(
    ggplot2::aes(ymin = lower, ymax = upper),
    width = 0.4,
    na.rm = TRUE
  ) +
  ggplot2::geom_text(ggplot2::aes(label = sprintf("%.3f", lower), y = lower), size = 2.25, hjust = 1.2, na.rm = TRUE) +
  ggplot2::geom_text(ggplot2::aes(label = sprintf("%.3f", upper), y = upper), size = 2.25, hjust = -0.2, na.rm = TRUE) +
  ggplot2::scale_color_manual(values = chat_model_colors, na.translate = FALSE) +
  ggplot2::scale_x_discrete(labels = model_labels) +
  ggplot2::scale_y_continuous(limits = c(0.65, 1.02), breaks = seq(0.7, 1.0, 0.1)) +
  ggplot2::coord_flip() +
  ggplot2::facet_wrap(~ Metric, ncol = 2, scales = "fixed", axes = 'all') +
  ggplot2::labs(
    title = "Classification Performance vs. StockNewsAPI Labels",
    subtitle = "<span style='color:#808080'>Cheaper</span> vs. <span style='color:#C3142D'>more expensive (time, cost) </span> LLMs by company",
    x = NULL,
    y = NULL
  ) +
  ggplot2::theme_minimal() +
  ggplot2::theme(
    plot.title = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
    plot.subtitle = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 8),
    axis.title.x = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 8),
    axis.title.y = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 8),
    axis.text.x = ggtext::element_markdown(hjust = 0.5, size = 7, face = 'bold'),
    axis.text.y = ggtext::element_markdown(hjust = 1, size = 7, face = 'bold'),
    strip.text = ggtext::element_markdown(face = "bold", size = 8),
    legend.position = "none",
    panel.grid = ggplot2::element_blank(),
    axis.line = ggplot2::element_line(color = "black"),
    axis.ticks = ggplot2::element_line(color = 'black')
  ) -> original_plot



# * Adding a table in the empty space at the bottom right -----------------

# Custom theme
booktabs_theme <- ttheme_minimal(
  core = list(
    fg_params = list(fontface = "plain", cex = 0.55, hjust = 0, x = 0.05)
  ),
  colhead = list(
    fg_params = list(fontface = "bold", cex = 0.6, hjust = 0, x = 0.05),
    bg_params = list(fill = NA)  # No shading, like booktabs
  ),
  padding = unit(c(1.7, 1.7), "mm")  # Tighter padding
)

# Base table
table_grob <- gridExtra::tableGrob(table_df, rows = NULL, theme = booktabs_theme)
table_grob$heights <- ggplot2::unit(rep(1, nrow(table_grob)), "lines") * 0.5

# Add title row
title_grob <- grid::textGrob(
  "Validity with Benchmark Model",
  gp = grid::gpar(fontface = "bold", fontsize = 9),
  x = 0.5, hjust = 0.5
)

# Add title row above the table
table_grob <- gtable::gtable_add_rows(table_grob, heights = unit(1.5, "lines"), pos = 0)
table_grob <- gtable::gtable_add_grob(
  table_grob,
  grobs = title_grob,
  t = 1, l = 1, r = ncol(table_grob)
)

table_width <- sum(table_grob$widths)
table_height <- sum(table_grob$heights)

padding <- ggplot2::unit(.5, "mm")

# Color code the table 
core_llm_indices <- which(
  table_grob$layout$name == "core-fg" & 
    table_grob$layout$l == 1  # column 1
)

model_colors <- rep(c("#808080", "#C3142D"), length.out = 14)  # matches inter_wide

for (i in seq_along(core_llm_indices)) {
  grob_index <- core_llm_indices[i]
  original_gp <- table_grob$grobs[[grob_index]]$gp
  table_grob$grobs[[grob_index]]$gp <- modifyList(original_gp, gpar(col = model_colors[i]))
}

# Add border around entire table using grobTree
bordered_table <- grid::grobTree(
  grid::rectGrob(
    width = table_width+ padding,
    height = table_height + padding,
    gp = grid::gpar(fill = NA, lwd = 0.7, col ='black') 
  ),
  table_grob
)


final_plot <- cowplot::ggdraw() +
  cowplot::draw_plot(original_plot, 0, 0, 1, 1) +  # main plot takes full area
  cowplot::draw_grob(bordered_table, x = 0.742, y = 0.16, width = 0.01, height = 0.01)

final_plot

4.6 Validity of the Binary Classification Labels (Comparison with “External Criterion” Impacts)

In this section, we will repeat our analysis from the previous section, but this time we will compare the LLM labels with the “External Criterion” effects to what actually happened to the stock prices. We will calculate the agreement between the LLM labels and the “External Criterion” labels using our agreement metrics.

4.6.1 Extracting the External Criterion Stock Performance

In the code chunk below, we use R and the tidyquant package to extract the stock prices for the tickers associated with the binary classification labels. We then save the raw stock prices to a CSV file for further analysis.

tickers_df = readr::read_csv(
  file = "../data/binary_classification_data.csv"
)

tickers_list = tickers_df$tickers |> unique()
min_date = min(tickers_df$date)
max_date = max(tickers_df$date)


stock_prices = tidyquant::tq_get(
  x = tickers_list,
  from = min_date,
  to = max_date,
  get = "stock.prices"
)

stock_prices$symbol |> unique() |> length()

other_stocks = c("MGC", "^GSPC")

#AHT, CRWV, MTTR (after 2025-03-03 not available)

other_prices = tidyquant::tq_get(
  x = other_stocks,
  from = min_date,
  to = max_date,
  get = "stock.prices"
) 

stock_prices |>
  dplyr::select(symbol, date, adjusted) |>
  dplyr::rename(ticker = symbol) |> 
  dplyr::bind_rows(other_prices) |>
  dplyr::arrange(ticker, date) |> 
  readr::write_csv(
    "../results/stock_prices_raw.csv"
  )

4.6.2 Computing the Leading News Effect

# read the stock prices and compute the returns
returns_df = 
  readr::read_csv("../results/stock_prices_raw.csv") |> 
  dplyr::group_by(ticker) |>
  dplyr::mutate(
    return = (adjusted - dplyr::lag(adjusted)) / dplyr::lag(adjusted),
    perc_return = return * 100
  ) |> 
  dplyr::ungroup()

# calculate the baseline s&p returns
sp500_returns = returns_df |>
  dplyr::filter(ticker == '^GSPC') |>
  dplyr::select(date, sp500_perc_return = perc_return)

# combine that with the stock returns
returns_df = returns_df |> 
  dplyr::left_join(sp500_returns, by = "date") |> 
  dplyr::filter(ticker != '^GSPC') |>
  dplyr::mutate(
    excess_return = perc_return - sp500_perc_return,
    excess_return_sign = dplyr::case_when(
      excess_return > 0 ~ "Positive",
      excess_return < 0 ~ "Negative",
      TRUE ~ NA
    ),
    leading_news_effect = dplyr::lead(excess_return_sign)
  )

# save the returns dataframe as RDS and CSV files
readr::write_rds(returns_df, "../results/stock_returns_df.rds")
readr::write_csv(returns_df, "../results/stock_returns_df.csv")

# show the results for the first 5 tickers of interest (excluding S&P 500)
DT::datatable(
  returns_df |> 
    dplyr::filter( ticker %in% unique(returns_df$ticker)[1:5] ), 
  rownames = FALSE, 
  extensions = c("FixedColumns"),
  options = list(
    pageLength = 10, 
    scrollX = TRUE,
    fixedColumns = list(leftColumns = 1)
  )
) |> 
  DT::formatRound(columns = 3:7, digits = 2) |> 
  DT::formatStyle(
    columns = 'leading_news_effect',  # Apply to the 9th column (news_effect)
    valueColumns = 'leading_news_effect',
    color = DT::JS("value == 'Positive' ? '#2C7BB6' : '#D7191C '"), 
    fontWeight = 'bold'       # Bold the text
  )

4.6.3 Merging the `News Effect` with Our Binary Classification Labels

In the code chunk below, we merge the binary classification labels with the leading news effect. We also calculate the distribution of the leading news effect and identify the tickers with missing values.

# read the stock returns dataframe
adjusted_news_df = 
  readr::read_csv("../results/binary_analysis_df.csv") |> 
  dplyr::mutate(date = lubridate::as_date(date)) |> 
  dplyr::select(-percent_agreement)

## Rows: 18900 Columns: 12
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr  (9): title, text, tickers, chat_model, rep_1, rep_2, rep_3, rep_4, rep_5
## dbl  (2): percent_agreement, percent_agreement_drop_na
## dttm (1): date
## 
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

# NYSE holidays within the date range
min_date = min(adjusted_news_df$date) |> as.Date()
max_date = max(adjusted_news_df$date) |> as.Date()

# nyse holidays within our range dates
nyse_holidays = 
   timeDate::holidayNYSE(year = lubridate::year(min_date):lubridate::year(max_date)) |> 
   as.data.frame() |> 
   dplyr::rename(off_dates = 1) |>
   dplyr::mutate(off_dates = as.Date(off_dates)) |> 
   dplyr::filter(off_dates >= min_date & off_dates <= max_date) |>  
   dplyr::pull(off_dates) 

# weekends within the date range
weekends =
  data.frame(off_dates = seq.Date(min_date, max_date, by = "day")) |>
  dplyr::mutate(
    weekday = lubridate::wday(off_dates, label = TRUE, abbr = FALSE)
    ) |> 
  dplyr::filter(weekday %in% c('Saturday', 'Sunday') ) |> 
  dplyr::pull(off_dates)

# combine the holidays and weekends
# off_dates = c(nyse_holidays, weekends) |> as.Date()
off_dates = weekends

# trading days
trading_days = seq.Date(min_date, max_date, by = "day")
trading_days_index = !trading_days %in% off_dates 
trading_days = trading_days[trading_days_index]

# helper function to find the next trading day after the current date
next_trading_day = function(current_date, trading_days) {
  next_day = trading_days[trading_days > current_date]
  if (length(next_day) > 0) {
    return(min(next_day))  # return the next available trading day
  } else {
    return(NA)  # if no future trading day is available (unlikely in this case)
  }
}

# create a binary gold dataframe with the leading news effect
binary_gold_df = 
  adjusted_news_df |> 
  dplyr::mutate(
    day_of_week = lubridate::wday(date, label = TRUE, abbr = FALSE),
    impact_date = date,  # set impact_date as original date first
    # if it's a holiday or weekend, pick the next available trading day
    impact_date = dplyr::if_else(
      date %in% off_dates, 
      purrr::map_dbl(date, ~ next_trading_day(.x, trading_days)) |> as.Date(),  
      impact_date  # Keep original date if not a weekend/holiday
    )
  ) |> 
  # left join with returns_df using the adjusted impact_date
  dplyr::left_join(
    returns_df |> dplyr::select(date, ticker, leading_news_effect),
    by = c("impact_date" = "date", "tickers" = "ticker")
  )

# distribution of the leading news effect
news_effect_table = 
  table(binary_gold_df$leading_news_effect, useNA = "ifany") / 14


# explanation with cat
cat(
  "The table below shows the average distribution of the leading news effect, normalized by the number of LLM models (14):",
  "\n\nNegative news effect count per model:", news_effect_table["Negative"],
  "\nPositive news effect count per model:", news_effect_table["Positive"],
  "\nMissing or NA values per model:", news_effect_table[3],
  "\n\nThe missing or NA values are likely due to the fact that we could not retrieve stock prices for certain tickers, possibly because they represent private stocks."
)

The table below shows the average distribution of the leading news effect, normalized by the number of LLM models (14):

Negative news effect count per model: 671 Positive news effect count per model: 672 Missing or NA values per model: 7

The missing or NA values are likely due to the fact that we could not retrieve stock prices for certain tickers, possibly because they represent private stocks.

4.6.4 Visualizing the Validity Metrics

The comparison against actual market behavior, depicted in the figure below, contrasts the benchmark results. The dot represents each metric’s mean value (from its intra-rater five replicates), and the whiskers length reflect the standard error.

# library(tidyverse)
# library(grid)
# library(gridExtra)

# ---- Setup ----
chat_models = c(
  "phi4-mini",
  "phi4:latest",
  "llama3.2:1B",
  "llama3.2:3B", 
  "gemma3:1B",
  "gemma3:27B",
  "deepseek-r1:1.5B",
  "deepseek-r1:7B",
  "command-r7b",
  "command-r-plus-08-2024",
  "gpt-4o-mini-2024-07-18",
  "gpt-4o-2024-11-20",
  "claude-3-5-haiku-20241022",
  "claude-3-7-sonnet-20250219"
)

model_abbrev = c(
  "phi4-mini"                 = "phi4-mini",
  "phi4:latest"              = "phi4:latest",
  "llama3.2:1B"              = "llama3.2:1B",
  "llama3.2:3B"              = "llama3.2:3B",
  "gemma3:1B"                = "gemma3:1B",
  "gemma3:27B"               = "gemma3:27B",
  "deepseek-r1:1.5B"         = "deepseek-r1:1.5B",
  "deepseek-r1:7B"           = "deepseek-r1:7B",
  "command-r7b"              = "command-r7b",
  "command-r-plus-08-2024"   = "command-r-plus",
  "gpt-4o-mini-2024-07-18"   = "gpt-4o-mini",
  "gpt-4o-2024-11-20"        = "gpt-4o",
  "claude-3-5-haiku-20241022"= "claude-3-5-haiku",
  "claude-3-7-sonnet-20250219" = "claude-3-7-sonnet"
)

chat_model_colors = rep(c('#808080', '#C3142D'), length.out = length(chat_models))
chat_model_colors = stats::setNames(chat_model_colors, model_abbrev[chat_models])


model_labels <- purrr::map_chr( model_abbrev[chat_models], ~{
  if (grepl("^spacer", .x)) return("")
  glue::glue("<span style='color:{chat_model_colors[.x]}'><b>{.x}</b></span>")
})
names(model_labels) <- model_abbrev[chat_models]




# ---- Load Data ----
inter_df = 
  tibble::tribble(
    ~Model,             ~Accuracy, ~Acc_SE, ~TPR,    ~TPR_SE, ~TNR,    ~TNR_SE, ~PPV,    ~PPV_SE, ~F1,    ~F1_SE,
    "phi4-mini",        0.487,     0.002,   0.495,   0.002,   0.479,   0.006,   0.485,   0.002,  0.490,   0.000,
    "phi4:latest",      0.513,     0.003,   0.526,   0.001,   0.500,   0.004,   0.511,   0.003,  0.518,   0.002,
    "llama3.2:1B",      0.512,     0.000,   0.527,   0.001,   0.497,   0.001,   0.510,   0.000,  0.518,   0.000,
    "llama3.2:3B",      0.478,     0.000,   0.525,   0.003,   0.432,   0.002,   0.478,   0.000,  0.501,   0.002,
    "gemma3:1B",        0.489,     0.002,   0.442,   0.005,   0.535,   0.001,   0.486,   0.002,  0.463,   0.004,
    "gemma3:27B",       0.509,     0.002,   0.505,   0.003,   0.512,   0.001,   0.507,   0.002,  0.506,   0.002,
    "deepseek-r1:1.5B", 0.493,     0.003,   0.447,   0.010,   0.539,   0.004,   0.490,   0.003,  0.468,   0.007,
    "deepseek-r1:7B",   0.496,     0.002,   0.509,   0.002,   0.483,   0.001,   0.494,   0.002,  0.501,   0.002,
    "command-r-plus",   0.512,     0.004,   0.450,   0.002,   0.574,   0.007,   0.511,   0.005,  0.479,   0.003,
    "command-r7b",      0.507,     0.004,   0.553,   0.002,   0.462,   0.005,   0.505,   0.003,  0.527,   0.003,
    "gpt-4o-mini",      0.502,     0.002,   0.482,   0.004,   0.522,   0.003,   0.500,   0.003,  0.491,   0.003,
    "gpt-4o",           0.509,     0.002,   0.461,   0.007,   0.558,   0.005,   0.508,   0.002,  0.483,   0.004,
    "claude-3-5-haiku", 0.512,     0.003,   0.549,   0.006,   0.476,   0.002,   0.509,   0.003,  0.528,   0.004,
    "claude-3-7-sonnet",0.514,     0.003,   0.548,   0.005,   0.480,   0.002,   0.511,   0.002,  0.529,   0.004
  )

inter_df = inter_df |> 
  dplyr::rename(Accuracy_mean = Accuracy,
                TPR_mean = TPR,
                TNR_mean = TNR,
                PPV_mean = PPV,
                F1_mean = F1) |>
  dplyr::rename(Accuracy_se = Acc_SE,
                TPR_se = TPR_SE,
                TNR_se = TNR_SE,
                PPV_se = PPV_SE,
                F1_se = F1_SE)


inter_long = inter_df |> 
  tidyr::pivot_longer(
    cols = -Model,
    names_to = c("Metric", ".value"),
    names_sep = "_"
  ) |>
  dplyr::mutate(
    lower = mean - (se/sqrt(5)),
    upper = mean + (se/sqrt(5)),
    Model = factor(Model, levels = rev(model_abbrev[chat_models]))
    )

table_df <- inter_df |>
  dplyr::select(Model, Accuracy_mean, TPR_mean, TNR_mean, PPV_mean, F1_mean) %>%
  dplyr::rename(
    LLM = Model,
    ACC = Accuracy_mean,
    TPR = TPR_mean,
    TNR = TNR_mean,
    PPV = PPV_mean,
    F1 = F1_mean) |>
  dplyr::mutate(dplyr::across(dplyr::where(is.numeric), ~ sprintf("%.2f", .))
  )


# *Plot the Data ----------------------------------------------------------

inter_long |> 
  ggplot2::ggplot(ggplot2::aes(x = Model, y = mean, color = Model)) +
  ggplot2::geom_blank() +
  ggplot2::geom_point(size = 0.9, na.rm = TRUE) +
  ggplot2::geom_errorbar(
    ggplot2::aes(ymin = lower, ymax = upper),
    width = 0.4,
    na.rm = TRUE
  ) +
  ggplot2::geom_text(ggplot2::aes(label = sprintf("%.3f", lower), y = lower), size = 2.25, hjust = 1.2, na.rm = TRUE) +
  ggplot2::geom_text(ggplot2::aes(label = sprintf("%.3f", upper), y = upper), size = 2.25, hjust = -0.2, na.rm = TRUE) +
  ggplot2::scale_color_manual(values = chat_model_colors, na.translate = FALSE) +
  ggplot2::scale_x_discrete(labels = model_labels) +
  ggplot2::scale_y_continuous(limits = c(0.375, 0.65), breaks = seq(0.4, .61, 0.1)) +
  ggplot2::coord_flip() +
  ggplot2::facet_wrap(~ Metric, ncol = 2, scales = "fixed", axes = 'all') +
  ggplot2::labs(
    title = "Classification Performance vs. Actual Market Behavior",
    subtitle = "<span style='color:#808080'>Cheaper</span> vs. <span style='color:#C3142D'>more expensive (time, cost) </span> LLMs by company",
    x = NULL,
    y = NULL
  ) +
  ggplot2::theme_minimal() +
  ggplot2::theme(
    plot.title = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 9),
    plot.subtitle = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 8),
    axis.title.x = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 8),
    axis.title.y = ggtext::element_markdown(hjust = 0.5, face = "bold", size = 8),
    axis.text.x = ggtext::element_markdown(hjust = 0.5, size = 7, face = 'bold'),
    axis.text.y = ggtext::element_markdown(hjust = 1, size = 7, face = 'bold'),
    strip.text = ggtext::element_markdown(face = "bold", size = 8),
    legend.position = "none",
    panel.grid = ggplot2::element_blank(),
    axis.line = ggplot2::element_line(color = "black"),
    axis.ticks = ggplot2::element_line(color = 'black')
  ) -> original_plot


# * Adding a table in the empty space at the bottom right -----------------

# Custom theme
booktabs_theme <- ttheme_minimal(
  core = list(
    fg_params = list(fontface = "plain", cex = 0.55, hjust = 0, x = 0.05)
  ),
  colhead = list(
    fg_params = list(fontface = "bold", cex = 0.6, hjust = 0, x = 0.05),
    bg_params = list(fill = NA)  # No shading, like booktabs
  ),
  padding = unit(c(1.7, 1.7), "mm")  # Tighter padding
)

# Base table
table_grob <- gridExtra::tableGrob(table_df, rows = NULL, theme = booktabs_theme)
table_grob$heights <- ggplot2::unit(rep(1, nrow(table_grob)), "lines") * 0.5

# Add title row
title_grob <- grid::textGrob(
  "Validity with External Criterion",
  gp = grid::gpar(fontface = "bold", fontsize = 9),
  x = 0.5, hjust = 0.5
)

# Add title row above the table
table_grob <- gtable::gtable_add_rows(table_grob, heights = unit(1.5, "lines"), pos = 0)
table_grob <- gtable::gtable_add_grob(
  table_grob,
  grobs = title_grob,
  t = 1, l = 1, r = ncol(table_grob)
)

table_width <- sum(table_grob$widths)
table_height <- sum(table_grob$heights)

padding <- ggplot2::unit(.5, "mm")

# Color code the table 
core_llm_indices <- which(
  table_grob$layout$name == "core-fg" & 
    table_grob$layout$l == 1  # column 1
)

model_colors <- rep(c("#808080", "#C3142D"), length.out = 14)  # matches inter_wide

for (i in seq_along(core_llm_indices)) {
  grob_index <- core_llm_indices[i]
  original_gp <- table_grob$grobs[[grob_index]]$gp
  table_grob$grobs[[grob_index]]$gp <- modifyList(original_gp, gpar(col = model_colors[i]))
}

# Add border around entire table using grobTree
bordered_table <- grid::grobTree(
  grid::rectGrob(
    width = table_width+ padding,
    height = table_height + padding,
    gp = grid::gpar(fill = NA, lwd = 0.7, col ='black') 
  ),
  table_grob
)


final_plot <- cowplot::ggdraw() +
  cowplot::draw_plot(original_plot, 0, 0, 1, 1) +  # main plot takes full area
  cowplot::draw_grob(bordered_table, x = 0.742, y = 0.16, width = 0.01, height = 0.01)

final_plot

Email: fmegahed@miamioh.edu | Phone: +1-513-529-4185 | Website: Miami University Official ↩︎
Email: ychen4@udayton.edu | Phone: +1-937-229-2405 | Website: University of Dayton Official ↩︎
Email: farmerl2@miamioh.edu | Phone: +1-513-529-4823 | Website: Miami University Official ↩︎
Email: leeyh2@miamioh.edu | Phone: +1-513-529-2164 | Website: Miami University Official ↩︎
Email: wangj249@miamioh.edu | Phone: +1-513-529-1577 | Website: Miami University Official ↩︎
Email: i.m.zwetsloot@uva.nl | Website: University of Amsterdam Official ↩︎

LS0tDQp0aXRsZTogIkV2YWx1YXRpbmcgdGhlIENvbnNpc3RlbmN5IG9mIE11bHRpcGxlIExMTXMgd2l0aCBJbnRyYSBhbmQgSW50ZXIgUmVsaWFiaWxpdHkgTWV0aG9kcyINCmF1dGhvcjoNCiAgLSBuYW1lOiAiRmFkZWwgTS4gTWVnYWhlZCBeW0VtYWlsOiBmbWVnYWhlZEBtaWFtaW9oLmVkdSB8IFBob25lOiArMS01MTMtNTI5LTQxODUgfCBXZWJzaXRlOiA8YSBocmVmPVwiaHR0cHM6Ly9taWFtaW9oLmVkdS9mc2IvZGlyZWN0b3J5Lz91cD0vZGlyZWN0b3J5L21lZ2FoZWZtXCI+TWlhbWkgVW5pdmVyc2l0eSBPZmZpY2lhbDwvYT5dIg0KICAgIGFmZmlsaWF0aW9uOiBGYXJtZXIgU2Nob29sIG9mIEJ1c2luZXNzLCBNaWFtaSBVbml2ZXJzaXR5DQogIC0gbmFtZTogIllpbmctSnUgKFRlc3NhKSBDaGVuIF5bRW1haWw6IHljaGVuNEB1ZGF5dG9uLmVkdSB8IFBob25lOiArMS05MzctMjI5LTI0MDUgfCBXZWJzaXRlOiA8YSBocmVmPVwiaHR0cHM6Ly91ZGF5dG9uLmVkdS9kaXJlY3RvcnkvYXJ0c3NjaWVuY2VzL21hdGhlbWF0aWNzL2NoZW4teWluZy1qdS5waHBcIj5Vbml2ZXJzaXR5IG9mIERheXRvbiBPZmZpY2lhbDwvYT5dIg0KICAgIGFmZmlsaWF0aW9uOiBEZXBhcnRtZW50IG9mIE1hdGhlbWF0aWNzLCBVbml2ZXJzaXR5IG9mIERheXRvbg0KICAtIG5hbWU6ICJBbGxpc29uIEpvbmVzLUZhcm1lciBeW0VtYWlsOiBmYXJtZXJsMkBtaWFtaW9oLmVkdSB8IFBob25lOiArMS01MTMtNTI5LTQ4MjMgfCBXZWJzaXRlOiA8YSBocmVmPVwiaHR0cHM6Ly9taWFtaW9oLmVkdS9mc2IvZGlyZWN0b3J5Lz91cD0vZGlyZWN0b3J5L2Zhcm1lcmwyXCI+TWlhbWkgVW5pdmVyc2l0eSBPZmZpY2lhbDwvYT5dIg0KICAgIGFmZmlsaWF0aW9uOiBGYXJtZXIgU2Nob29sIG9mIEJ1c2luZXNzLCBNaWFtaSBVbml2ZXJzaXR5DQogIC0gbmFtZTogIkdhYmUgTGVlIF5bRW1haWw6IGxlZXloMkBtaWFtaW9oLmVkdSB8IFBob25lOiArMS01MTMtNTI5LTIxNjQgfCBXZWJzaXRlOiA8YSBocmVmPVwiaHR0cHM6Ly9taWFtaW9oLmVkdS9mc2IvZGlyZWN0b3J5Lz91cD0vZGlyZWN0b3J5LyBsZWV5aDJcIj5NaWFtaSBVbml2ZXJzaXR5IE9mZmljaWFsPC9hPl0iDQogICAgYWZmaWxpYXRpb246IEZhcm1lciBTY2hvb2wgb2YgQnVzaW5lc3MsIE1pYW1pIFVuaXZlcnNpdHkNCiAgLSBuYW1lOiAiQnJvb2tlIFdhbmcgXltFbWFpbDogd2FuZ2oyNDlAbWlhbWlvaC5lZHUgfCBQaG9uZTogKzEtNTEzLTUyOS0xNTc3IHwgV2Vic2l0ZTogPGEgaHJlZj1cImh0dHBzOi8vbWlhbWlvaC5lZHUvZnNiL2RpcmVjdG9yeS8/dXA9L2RpcmVjdG9yeS93YW5najI0OVwiPk1pYW1pIFVuaXZlcnNpdHkgT2ZmaWNpYWw8L2E+XSINCiAgICBhZmZpbGlhdGlvbjogRmFybWVyIFNjaG9vbCBvZiBCdXNpbmVzcywgTWlhbWkgVW5pdmVyc2l0eQ0KICAtIG5hbWU6ICJJbmV6IE0uIFp3ZXRzbG9vdCBeW0VtYWlsOiBpLm0uendldHNsb290QHV2YS5ubCB8IFdlYnNpdGU6IDxhIGhyZWY9XCJodHRwczovL3d3dy51dmEubmwvZW4vcHJvZmlsZS96L3cvaS5tLnp3ZXRzbG9vdC9pLm0uendldHNsb290Lmh0bWw/Y2JcIj5Vbml2ZXJzaXR5IG9mIEFtc3RlcmRhbSBPZmZpY2lhbDwvYT5dIg0KICAgIGFmZmlsaWF0aW9uOiBGYWN1bHR5IG9mIEVjb25vbWljcyBhbmQgQnVzaW5lc3MsIFVuaXZlcnNpdHkgb2YgQW1zdGVyZGFtDQpkYXRlOiAiYHIgZm9ybWF0KFN5cy50aW1lKCksICclQiAlZCwgJVknKWAiDQpvdXRwdXQ6IA0KICBodG1sX2RvY3VtZW50Og0KICAgIHRvYzogVFJVRQ0KICAgIHRvY19mbG9hdDogVFJVRQ0KICAgIG51bWJlcl9zZWN0aW9uczogVFJVRQ0KICAgIGNvZGVfZm9sZGluZzogc2hvdw0KICAgIGNvZGVfZG93bmxvYWQ6IFRSVUUNCiAgICB0aGVtZTogc2ltcGxleA0KLS0tDQoNCg0KYGBge3Igc2V0dXAsIGluY2x1ZGU9RkFMU0V9DQprbml0cjo6b3B0c19jaHVuayRzZXQoZWNobyA9IFRSVUUpDQpgYGANCg0KDQojIFByZWZhY2Ugey19DQoNCkluIHRoaXMgcHJvamVjdCwgd2UgdXRpbGl6ZSB0aGUgb3Blbi1zb3VyY2UgYHIgZm9udGF3ZXNvbWU6OmZhKG5hbWUgPSAici1wcm9qZWN0IiwgZmlsbCA9ICJzdGVlbGJsdWUiKWAgcHJvZ3JhbW1pbmcgbGFuZ3VhZ2UsIGFsb25nc2lkZSBQeXRob24sIHRvIGV2YWx1YXRlIHRoZSBjb25zaXN0ZW5jeSBvZiBsYXJnZSBsYW5ndWFnZSBtb2RlbHMgKExMTXMpIHRocm91Z2ggaW50cmEtIGFuZCBpbnRlci1yZWxpYWJpbGl0eSBtZXRob2RzLiBGb3IgYHIgZm9udGF3ZXNvbWU6OmZhKG5hbWUgPSAici1wcm9qZWN0IiwgZmlsbCA9ICJzdGVlbGJsdWUiKWAsIHdlIGFyZSB1c2luZyAqdmVyc2lvbiA0LjQuMiAoMjAyNC0xMC0zMSB1Y3J0KSosIGFuZCB3ZSBtYW5hZ2UgcGFja2FnZSBkZXBlbmRlbmNpZXMgd2l0aCBgcmVudmAuIFRoZSBgcmVudmAgcGFja2FnZSBhbGxvd3MgZm9yIGlzb2xhdGVkIHByb2plY3QgZW52aXJvbm1lbnRzLCBlbnN1cmluZyB0aGUgc2FtZSB2ZXJzaW9ucyBvZiBgciBmb250YXdlc29tZTo6ZmEobmFtZSA9ICJyLXByb2plY3QiLCBmaWxsID0gInN0ZWVsYmx1ZSIpYCBwYWNrYWdlcyBhcmUgdXNlZCBjb25zaXN0ZW50bHkuIElmIHlvdSdyZSBuZXcgdG8gYHJlbnZgLCBpdCBlbmFibGVzIHRoZSBjcmVhdGlvbiBvZiAic25hcHNob3RzIiBmb3IgcGFja2FnZSB2ZXJzaW9ucywgZW5zdXJpbmcgcmVwcm9kdWNpYmlsaXR5IGFjcm9zcyBkaWZmZXJlbnQgc3lzdGVtcy4gWW91IGNhbiBmaW5kIGFsbCBwYWNrYWdlIHZlcnNpb25zIHVzZWQgaW4gdGhpcyBwcm9qZWN0IGluIG91ciBgcmVudmAgcmVwb3NpdG9yeSBhdCBbdGhpcyBsaW5rXShodHRwczovL2dpdGh1Yi5jb20vZm1lZ2FoZWQvbGxtX2NvbnN0aXRlbmN5L3RyZWUvbWFpbi9yZW52KS4NCg0KVGhpcyBkb2N1bWVudCBpcyBzdHJ1Y3R1cmVkIHVzaW5nIFJNYXJrZG93biwgd2hpY2ggc2VhbWxlc3NseSBpbnRlZ3JhdGVzIGJvdGggYHIgZm9udGF3ZXNvbWU6OmZhKG5hbWUgPSAici1wcm9qZWN0IiwgZmlsbCA9ICJzdGVlbGJsdWUiKWAgYW5kIFB5dGhvbiBjb2RlIGNodW5rcy4gUmVhZGVycyBjYW4gdGVsbCB0aGUgZGlmZmVyZW5jZSBiZXR3ZWVuIGByIGZvbnRhd2Vzb21lOjpmYShuYW1lID0gInItcHJvamVjdCIsIGZpbGwgPSAic3RlZWxibHVlIilgIGFuZCBQeXRob24gY29kZSBjaHVua3MgYnkgZG93bmxvYWRpbmcgdGhlIHJhdyBSTWFya2Rvd24gZmlsZSBmcm9tIFt0aGlzIHJlcG9zaXRvcnldKGh0dHBzOi8vZ2l0aHViLmNvbS9mbWVnYWhlZC9sbG1fY29uc2lzdGVuY3kvdHJlZS9tYWluL3JtYXJrZG93bikuIEluIGFkZGl0aW9uLCB0aGUgZG9jdW1lbnQgZmVhdHVyZXMgYSAqKmZsb2F0aW5nIHRhYmxlIG9mIGNvbnRlbnRzIChUT0MpKiogb24gdGhlIHNpZGUsIHdoaWNoIG1ha2VzIG5hdmlnYXRpb24gdGhyb3VnaCB0aGUgc2VjdGlvbnMgbW9yZSBjb252ZW5pZW50LiBUaGUgVE9DIGZvbGxvd3MgdGhlIHJlYWRlciBhcyB0aGV5IHNjcm9sbCwgYWxsb3dpbmcgcXVpY2sgYWNjZXNzIHRvIGRpZmZlcmVudCBzZWN0aW9ucy4NCg0KV2XigJl2ZSBhbHNvIGltcGxlbWVudGVkICoqZm9sZGFibGUgY29kZSBjaHVua3MqKiwgZW5hYmxpbmcgdGhlIHVzZXIgdG8gZXhwYW5kIGFuZCBjb2xsYXBzZSBjb2RlIGFzIG5lZWRlZCB0byBmb2N1cyBvbiB0aGUgZXhwbGFuYXRpb25zIG9yIHJlc3VsdHMuIFRoaXMgZmVhdHVyZSBpbXByb3ZlcyB0aGUgZG9jdW1lbnTigJlzIHJlYWRhYmlsaXR5IGJ5IGFsbG93aW5nIHlvdSB0byBoaWRlIHRoZSBjb2RlIHdoaWxlIHJlYWRpbmcgdGhlIG1haW4gY29udGVudC4gSWYgeW91IHdvdWxkIGxpa2UgdG8gdmlldyBvciBtb2RpZnkgdGhlIHJhdyBjb2RlLCBjbGljayB0aGUgIkNvZGUiIGJ1dHRvbiBhdCB0aGUgdG9wIG9mIHRoZSBkb2N1bWVudCB0byBkb3dubG9hZCB0aGUgZnVsbCBSTWFya2Rvd24gZmlsZS4gVGhpcyBnaXZlcyB5b3UgYWNjZXNzIHRvIGFsbCB0aGUgY29kZSBjaHVua3MgdXNlZCBpbiB0aGUgYW5hbHlzaXMuDQoNCkZvciB0aGUgUHl0aG9uIHNlY3Rpb25zLCB3ZSBhcmUgdXNpbmcgYSBjb25kYSBlbnZpcm9ubWVudCB3aXRoICpQeXRob24gdmVyc2lvbiAzLjEyLjkqLiBUaGUgcmVxdWlyZWQgUHl0aG9uIHBhY2thZ2VzIGNhbiBiZSBpbnN0YWxsZWQgdmlhIHRoZSBgcmVxdWlyZW1lbnRzLnR4dGAgZmlsZSwgYXZhaWxhYmxlIFtoZXJlXShodHRwczovL2dpdGh1Yi5jb20vZm1lZ2FoZWQvbGxtX2NvbnNpc3RlbmN5L2Jsb2IvbWFpbi9yZXF1aXJlbWVudHMudHh0KS4gVGhlIFJNYXJrZG93biBkb2N1bWVudCBsaW5rcyB0byB0aGUgY29uZGEgZW52aXJvbm1lbnQsIGFsbG93aW5nIFB5dGhvbiBjaHVua3MgdG8gZXhlY3V0ZSBhbG9uZ3NpZGUgdGhlIGByIGZvbnRhd2Vzb21lOjpmYShuYW1lID0gInItcHJvamVjdCIsIGZpbGwgPSAic3RlZWxibHVlIilgIGNvZGUgaW4gdGhlIHNhbWUgd29ya2Zsb3cuDQoNClRvIHNlY3VyZWx5IHN0b3JlIEFQSSBzZWNyZXRzIGZvciB0aGUgbGFyZ2UgbGFuZ3VhZ2UgbW9kZWxzIChMTE1zKSBhY2Nlc3NlZCBpbiB0aGUgUHl0aG9uIGNodW5rcywgd2UgdXNlIGEgYC5lbnZgIGZpbGUgdGhhdCBob2xkcyB0aGVzZSBrZXlzIGluIGEgc2FmZSwgZW52aXJvbm1lbnQtc3BlY2lmaWMgbWFubmVyLiBTaW1pbGFybHksIGZvciB0aGUgc3RvY2sgbmV3cyBBUEkgYWNjZXNzZWQgaW4gdGhlIGByIGZvbnRhd2Vzb21lOjpmYShuYW1lID0gInItcHJvamVjdCIsIGZpbGwgPSAic3RlZWxibHVlIilgIHNlY3Rpb25zLCB3ZSBzYXZlIHRoZSBBUEkgc2VjcmV0IGluIGEgcHJvamVjdC1iYXNlZCBgLlJlbnZpcm9uYCBmaWxlLiBCb3RoIGZpbGVzIGtlZXAgc2Vuc2l0aXZlIGNyZWRlbnRpYWxzIHNlY3VyZSBhbmQgYWxsb3cgdXMgdG8gYXZvaWQgaGFyZGNvZGluZyBBUEkgc2VjcmV0cyBkaXJlY3RseSBpbiB0aGUgY29kZS4gRm9yIG9idmlvdXMgc2VjdXJpdHkgcmVhc29ucywgd2UgZGlkIG5vdCBwdXNoIHRoZXNlIGAuZW52YCBhbmQgYC5SZW52aXJvbmAgZmlsZXMgdG8gb3VyIEdpdEh1YiByZXBvc2l0b3J5Lg0KDQpBbGwgaW5wdXQgYW5kIG91dHB1dCBmaWxlcyBmb3IgdGhpcyBwcm9qZWN0IGNhbiBiZSBhY2Nlc3NlZCBhbmQgZG93bmxvYWRlZCBmcm9tIG91ciBbR2l0SHViIHJlcG9zaXRvcnldKGh0dHBzOi8vZ2l0aHViLmNvbS9mbWVnYWhlZC9sbG1fY29uc2lzdGVuY3kpLg0KDQoNCiMgT2JqZWN0aXZlcyBvZiB0aGlzIEFuYWx5c2lzIHsjb2JqZWN0aXZlc30NCg0KLSBXZSBwcmVzZW50IG1pbmltdW0gc2FtcGxlIHNpemUgY2FsY3VsYXRpb25zIGZvciBvdXIgYmluYXJ5IGNsYXNzaWZpY2F0aW9uIGV4cGVyaW1lbnRzLiAgDQotIFdlIGRlc2NyaWJlIHRoZSBleHBlcmltZW50YWwgc2V0dXAgZm9yIHRoZSBjb25zaXN0ZW5jeSBhbmFseXNpcyBvZiBtdWx0aXBsZSBMTE1zLiAgDQotIFdlIHByb3ZpZGUgYSBjdXN0b20gZnVuY3Rpb24gZm9yIExMTSBjbGFzc2lmaWNhdGlvbiBleHBlcmltZW50cy4gIA0KLSBXZSBwcmVzZW50IGFuIGV4YW1wbGUgb2YgYSBiaW5hcnkgY2xhc3NpZmljYXRpb24gZXhwZXJpbWVudCwgdXNpbmcgdGhlIExMTXMgdG8gbGFiZWwgdGhlIHNlbnRpbWVudCBvZiBuZXdzIGFydGljbGVzIHJlbGF0ZWQgdG8gdGhlIHN0b2NrIG1hcmtldCwgYW5kIHV0aWxpemUgdGhlIGludHJhLSBhbmQgaW50ZXItcmF0ZXIgcmVsaWFiaWxpdHkgbWV0aG9kcyB0byBldmFsdWF0ZSB0aGUgY29uc2lzdGVuY3kgb2YgdGhlIExMTXMuDQoNCg0KDQojIFNhbXBsZSBTaXplIENhbGN1bGF0aW9ucyBmb3Igb3VyIEJpbmFyeSBDbGFzc2lmaWNhdGlvbiBFeHBlcmltZW50IHsjc2FtcGxlX3NpemV9DQoNClRoZSBtaW5pbXVtIHNhbXBsZSBzaXplIGZvciBlYWNoIG9mIG91ciBleHBlcmltZW50cyB3YXMgY29tcHV0ZWQgZm9yOiAqKnNpbXBsZSBwZXJjZW50IGFncmVlbWVudCoqLCAqKkd3ZXQncyBBQzEgY29lZmZpY2llbnQqKiwgYW5kICoqQnJlbm5hbi1QcmVkaWdlciBjb2VmZmljaWVudCoqLiBUaGUgbWluaW11bSBzYW1wbGUgc2l6ZSB3YXMgY29tcHV0ZWQgdXNpbmcgdGhlIHRhYmxlcyBpbiBbSGFuZGJvb2sgb2YgSW50ZXItUmF0ZXIgUmVsaWFiaWxpdHksIDV0aCBFZGl0aW9uLiBWb2x1bWUgMTogQW5hbHlzaXMgb2YgQ2F0ZWdvcmljYWwgUmF0aW5nc10oaHR0cHM6Ly9zaXRlcy5mYXN0c3ByaW5nLmNvbS9hZ3JlZXN0YXQvaW5zdGFudC9jYWM1ZWQ5NzhfMV83OTIzXzU0NjNfMmUpLiBUaGUgc2FtcGxlIHNpemVzIHdlcmUgY29tcHV0ZWQgZm9yIHRoZSB0aHJlZSBkaWZmZXJlbnQgbWV0cmljcywgd2l0aCBhIG1hcmdpbiBvZiBlcnJvciBvZiAwLjA1LCBhIGNvbmZpZGVuY2UgbGV2ZWwgb2YgMC45MCwgYW5kIGZvciBmaXZlIHJlcGxpY2F0ZXMuDQoNCmBgYHtyIGJpbmFyeV9jbGFzc2lmaWNhdGlvbl9zYW1wbGVfc2l6ZSwgY2xhc3Muc291cmNlID0gJ2ZvbGQtaGlkZScsIHJlc3VsdHM9J2FzaXMnfQ0KIyBjcmVhdGUgYSBkYXRhIGZyYW1lIG9mIHRoZSBvYnRhaW5lZCBzYW1wbGUgc2l6ZSByZXN1bHRzDQpiaW5hcnlfZGF0YSA9IGRhdGEuZnJhbWUoDQogIG1ldHJpYyA9IGMoIlBlcmNlbnQgQWdyZWVtZW50IiwgIkd3ZXTigJlzIEFDMSBDb2VmZmljaWVudCIsICJCcmVubmFuLVByZWRpZ2VyIENvZWZmaWNpZW50IiksDQogIGBzYW1wbGUgc2l6ZWAgPSBjKDIxNiwgMTMxNywgODQ3KSB8PiBzY2FsZXM6OmNvbW1hKCkNCikNCg0KIyBnZW5lcmF0ZSB0aGUgdGFibGUgaW4gSFRNTCBmb3JtYXQNCmtuaXRyOjprYWJsZSgNCiAgYmluYXJ5X2RhdGEsIGZvcm1hdCA9ICJodG1sIiwgdGFibGUuYXR0ciA9ICJzdHlsZT0nd2lkdGg6NTAlOyciLCANCiAgYWxpZ24gPSBjKCdsJywncicpDQogICkgfD4NCiAga2FibGVFeHRyYTo6a2FibGVfc3R5bGluZyhmdWxsX3dpZHRoID0gRkFMU0UpIHw+DQogIGthYmxlRXh0cmE6OmNvbHVtbl9zcGVjKDEsIGJvbGQgPSBUUlVFKQ0KYGBgDQoNClRoZXJlZm9yZSwgdXNpbmcgdGhlIGhpZ2hlc3Qgc2FtcGxlIHNpemUgYW1vbmcgdGhlIHRocmVlIG1ldHJpY3MsIHdlIG5lZWQgYXQgKipsZWFzdCAxLDMxNyBzYW1wbGVzKiogZm9yIHRoZSBiaW5hcnkgY2xhc3NpZmljYXRpb24gZXhwZXJpbWVudC4NCg0KDQoNCiMgRXhwZXJpbWVudGFsIFNldHVwIGZvciB0aGUgQ29uc2lzdGVuY3kgQW5hbHlzaXMgeyNleHBlcmltZW50YWxfc2V0dXB9DQoNCiMjIExMTXMgVXNlZCBhbmQgdGhlaXIgQVBJIEtleXMgeyNsbG1fc2V0dXB9DQoNCkluIG91ciBleHBlcmltZW50cywgd2Ugc2VsZWN0ZWQgdGhlIGZvbGxvd2luZyBMTE1zOg0KDQoqKkNsb3NlZCBzb3VyY2UgcHJvcHJpZXRhcnkgZm91bmRhdGlvbiBtb2RlbHM6KioNCg0KLSBgY2xhdWRlLTMtNy1zb25uZXQtMjAyNTAyMTlgLCB3aGljaCBpcyBbQW50aHJvcGljJ3MgY3VycmVudCBiZXN0IG1vZGVsXShodHRwczovL3d3dy5hbnRocm9waWMuY29tL25ld3MvY2xhdWRlLTMtNy1zb25uZXQpIGFzIG9mIE1hcmNoIDEyLCAyMDI1OyAgDQotYGNsYXVkZS0zLTUtaGFpa3UtMjAyNDEwMjJgLCB3aGljaCByZXByZXNlbnRzIHRoZSBzbWFsbGVzdCBtb2RlbCBwcm92aWRlZCBieSBbQW50aHJvcGljXShodHRwczovL2RvY3MuYW50aHJvcGljLmNvbS9lbi9kb2NzL2Fib3V0LWNsYXVkZS9tb2RlbHMjbW9kZWwtbmFtZXMpIGFzIG9mIE1hcmNoIDEyLCAyMDI1OyAgDQoNCg0KLSBgZ3B0LTRvLTIwMjQtMTEtMjBgLCB3aGljaCBpcyBbT3BlbkFJJ3MgbGF0ZXN0IHNuYXBzaG90IG9mIHRoZSA0byBzZXJpZXNdKGh0dHBzOi8vcGxhdGZvcm0ub3BlbmFpLmNvbS9kb2NzL21vZGVscy9ncHQtNG8pIGFzIG9mIE1hcmNoIDEyLCAyMDI1LiBOb3RlIHRoYXQgd2UgZG8gbm90IGV4YW1pbmUgdGhlIG1vcmUgcmVjZW50IGFuZCBjYXBhYmxlIFtHUFQtNC41IFByZXZpZXcgTW9kZWxdKGh0dHBzOi8vcGxhdGZvcm0ub3BlbmFpLmNvbS9kb2NzL21vZGVscy9ncHQtNC41LXByZXZpZXcpIGFzIHRoZXkgYXJlIHZlcnkgZXhwZW5zaXZlICgkMTUwIHBlciBtaWxsaW9uIG91dHB1dCB0b2tlbnMgdmVyc3VzIHRoZSBcJDEwIHBlciBtaWxsaW9uIG91dHB1dCB0b2tlbnMgZm9yIHRoZSBHUFQtNG8gc2VyaWVzKTsgIA0KLSBgZ3B0LTRvLW1pbmktMjAyNC0wNy0xOGAsIHdoaWNoIGlzIFtPcGVuQUkncyBsYXRlc3Qgc3RhYmxlIGFuZCBtaW5pIG1vZGVsXShodHRwczovL3BsYXRmb3JtLm9wZW5haS5jb20vZG9jcy9tb2RlbHMvZ3B0LTRvLW1pbmkpIGFzIG9mIE1hcmNoIDEyLCAyMDI1LiBUaGVzZSBtb2RlbHMgYXJlIGZhc3QgYW5kIGFyZSBjb3N0IG9wdGltaXplZCBmb3IgbG93LWxhdGVuY3kgYXBwbGljYXRpb25zOyAgDQoNCg0KKipPcGVuIExMTXMqKg0KDQpUaGUgT3BlbiBMTE1zIHdlcmUgcnVuIGxvY2FsbHkgdXNpbmcgdGhlIE9sbGFtYSBpbnRlcmZhY2UgKHYgMC42LjApLCB3aXRoIHRoZSBleGNlcHRpb24gb2YgdGhlIGBjb21tYW5kLXItcGx1cy0wOC0yMDI0YCB3aGljaCB3ZSBydW4gdXNpbmcgdGhlIENvaGVyZSBBUEkgYXMgaXRzIDEwNEIgcGFyYW1ldGVycyB3b3VsZCBiZSB0b28gbGFyZ2UgdG8gcnVuIG9uIG91ciBHUFUgKE5WSURJQSBSVFggNTAwMCBBZGEgR2VuZXJhdGlvbiwgd2l0aCAzMkdCIG9mIEdyYXBoaWNzIE1lbW9yeSkuIFRoZSBtb2RlbHMgYXJlOg0KDQotIGBjb21tYW5kLXItcGx1cy0wOC0yMDI0YCwgd2hpY2ggaXMgW0NvaGVyZSdzIGxhdGVzdCBsYXJnZSBsYW5ndWFnZSBtb2RlbF0oaHR0cHM6Ly9kb2NzLmNvaGVyZS5jb20vZG9jcy9jb21tYW5kLXItcGx1cykgYXMgb2YgTWFyY2ggMTIsIDIwMjU7ICANCi0gYGNvbW1hbmQtcjdiYCwgd2hpY2ggaXMgW0NvaGVyZSdzIGxhdGVzdCBzbWFsbCBsYW5ndWFnZSBtb2RlbF0oaHR0cHM6Ly9kb2NzLmNvaGVyZS5jb20vZG9jcy9jb21tYW5kLXI3YikgYXMgb2YgTWFyY2ggMTIsIDIwMjUuICoqTm90ZSB0aGF0IHdlIGFyZSBydW5uaW5nIHRoaXMgN0IgcGFyYW1ldGVyIG1vZGVsIGxvY2FsbHksIHZpYSB0aGUgT2xsYW1hIGludGVyZmFjZS4qKjsgIA0KDQotIGBkZWVwc2Vlay1yMTo3QmAsIHdoaWNoIGlzIGEgcG9wdWxhciBtb2RlbCBmcm9tIFtEZWVwU2Vla10oaHR0cHM6Ly9vbGxhbWEuY29tL2xpYnJhcnkvZGVlcHNlZWstcjEpIGFuZCBpcyB0aGUgZGVmYXVsdCB3aGVuIHB1bGxpbmcgdGhlIGBkZWVwc2Vlay1yMWAgbW9kZWxzIGZyb20gT2xsYW1hOw0KLSBgZGVlcHNlZWstcjE6MS41QmAsIHdoaWNoIGlzIHRoZSBzbWFsbGVzdCBtb2RlbCBmcm9tIFtEZWVwU2Vla10oaHR0cHM6Ly9vbGxhbWEuY29tL2xpYnJhcnkvZGVlcHNlZWstcjEpOyAgDQoNCjwhLS0gLSBgZXhhb25lLWRlZXA6Ny44YmAsIHdoaWNoIGlzIHRoZSBtZWRpdW0tc2l6ZWQgb3Blbi1zb3VyY2VkIEVYQU9ORSBtb2RlbCBmcm9tIExHIEFJLCB3aGljaCB3YXMgcmVsZWFzZWQgb24gTWFyY2ggMTksIDIwMjU7ICAgLS0+DQo8IS0tIC0gYGV4YW9uZS1kZWVwOjIuNGJgLCB3aGljaCBpcyB0aGUgc21hbGxlc3Qgb3Blbi1zb3VyY2VkIEVYQU9ORSBtb2RlbCBmcm9tIExHIEFJLCB3aGljaCB3YXMgcmVsZWFzZWQgb24gTWFyY2ggMTksIDIwMjU7IC0tPg0KDQotIGBnZW1tYTM6MjdCYCwgd2hpY2ggaXMgdGhlIGxhcmdlc3Qgb3Blbi1zb3VyY2VkIEdlbW1hMyBtb2RlbCBmcm9tIEdvb2dsZSwgd2hpY2ggd2FzIHJlbGVhc2VkIG9uIE1hcmNoIDEyLCAyMDI1Ow0KLSBgZ2VtbWEzOjFCYCwgd2hpY2ggaXMgdGhlIHNtYWxsZXN0IG9wZW4tc291cmNlZCBHZW1tYTMgbW9kZWwgZnJvbSBHb29nbGUsIHdoaWNoIHdhcyByZWxlYXNlZCBvbiBNYXJjaCAxMiwgMjAyNTsgIA0KDQotIGBsbGFtYTMuMjozQmAsIE1ldGEgd2VudCBzbWFsbCB3aXRoIGl0cyBvcGVuLXNvdXJjZWQgTGxhbWEzLjIgbW9kZWw7IHRoaXMgaXMgdGhlIGxhcmdlc3Qgb2YgdGhlIHR3byBtb2RlbHMgdGhhdCB0aGV5IHJlbGVhc2VkIGFzIHBhcnQgb2YgdGhlIExsYW1hMy4yIHNlcmllczsgIA0KLSBgbGxhbWEzLjI6MUJgLCBNZXRhIHdlbnQgc21hbGwgd2l0aCBpdHMgb3Blbi1zb3VyY2VkIExsYW1hMy4yIG1vZGVsOyB0aGlzIGlzIHRoZSBzbWFsbGVzdCBvZiB0aGUgdHdvIG1vZGVscyB0aGF0IHRoZXkgcmVsZWFzZWQgYXMgcGFydCBvZiB0aGUgTGxhbWEzLjIgc2VyaWVzOw0KDQotIGBwaGk0OmxhdGVzdGAsIFBoaS00IGlzIGEgMTRCIHBhcmFtZXRlciwgc3RhdGUtb2YtdGhlLWFydCBvcGVuIG1vZGVsIGZyb20gTWljcm9zb2Z0LjsNCi0gYHBoaTQtbWluaWAsIHdoaWNoIGlzIHRoZSBzbWFsbGVzdCBvcGVuLXNvdXJjZWQgUGhpIE1vZGVsIGZyb20gTWljcm9zb2Z0IHdpdGggMy44QiBwYXJhbWV0ZXJzOw0KDQoNCmBgYHtweXRob24gbW9kZWxzLCBldmFsPUZBTFNFfQ0KIyBsb2FkIHRoZSBuZWNlc3NhcnkgbGlicmFyaWVzDQpmcm9tIGRvdGVudiBpbXBvcnQgbG9hZF9kb3RlbnYsIGZpbmRfZG90ZW52DQpmcm9tIGxhbmdjaGFpbi5wcm9tcHRzLmNoYXQgaW1wb3J0IENoYXRQcm9tcHRUZW1wbGF0ZQ0KDQojIEZpbmQgdGhlIC5lbnYgcGF0aA0KZG90ZW52X3BhdGggPSBmaW5kX2RvdGVudigpDQoNCiMgbG9hZCB0aGUgZW52aXJvbm1lbnQgdmFyaWFibGVzDQpsb2FkX2RvdGVudihkb3RlbnZfcGF0aCwgb3ZlcnJpZGU9VHJ1ZSkNCg0KIyB0aGUgQVBJIGtleXMgZm9yIHRoZSBkaWZmZXJlbnQgTExNcw0Kb3BlbmFpX2FwaV9rZXkgPSBvcy5nZXRlbnYoJ09QRU5BSV9BUElfS0VZJykNCmFudGhyb3BpY19hcGlfa2V5ID0gb3MuZ2V0ZW52KCdBTlRIUk9QSUNfQVBJX0tFWScpDQpjb2hlcmVfYXBpX2tleSA9IG9zLmdldGVudignQ09IRVJFX0FQSV9LRVknKQ0KDQoNCiMgdGhlIExMTSBtb2RlbHMgdG8gYmUgdXNlZCBmb3IgbGFiZWxpbmcNCiMgd2UgcmFuIHRoaXMgb3ZlciB0d28gZGlmZmVyZW50IHJ1biB0aW1lcyBzaW5jZSB3ZQ0KIyBoYWQgdG8gYWJvcnQgdGhlICJleGFvbmUtZGVlcCIgbW9kZWxzIGFzIHRoZXkgd2VyZSBxdWl0ZSBzbG93DQojIG90aGVyIGNvbW1lbnRlZCBvdXQgbW9kZWxzIHdlcmUgc3VjY2Vzc2Z1bGx5IHJ1biBpbiB0aGUgZmlyc3QgcnVuDQojIHNvIHdlIGRpZCBub3QgbmVlZCB0byBydW4gdGhlbSBhZ2Fpbg0KbW9kZWxzID0gWw0KICAjICdjbGF1ZGUtMy03LXNvbm5ldC0yMDI1MDIxOScsDQogICMgJ2NsYXVkZS0zLTUtaGFpa3UtMjAyNDEwMjInLA0KICANCiAgJ2dwdC00by0yMDI0LTExLTIwJywNCiAgJ2dwdC00by1taW5pLTIwMjQtMDctMTgnLA0KICANCiAgIyAnY29tbWFuZC1yLXBsdXMtMDgtMjAyNCcsDQogICMgJ2NvbW1hbmQtcjdiJywNCiAgDQogICMgJ2RlZXBzZWVrLXIxOjdCJywNCiAgIyAnZGVlcHNlZWstcjE6MS41QicsDQogIA0KICAjICdleGFvbmUtZGVlcDo3LjhiJywNCiAgIyAnZXhhb25lLWRlZXA6Mi40YicsDQogIA0KICAnZ2VtbWEzOjI3QicsDQogICdnZW1tYTM6MUInLA0KICANCiAgJ2xsYW1hMy4yOjNCJywNCiAgJ2xsYW1hMy4yOjFCJywNCiAgDQogICdwaGk0OmxhdGVzdCcsDQogICdwaGk0LW1pbmknDQogIF0NCmBgYA0KDQojIyBBIEN1c3RvbSBGdW5jdGlvbiBmb3IgTExNIENhbGxzIHsjY3VzdG9tX2xsbV9mdW5jdGlvbn0NCg0KV2UgY3JlYXRlZCBhIGN1c3RvbSBmdW5jdGlvbiwgYGdlbmVyYWxpemVkX2NoYXRfY29tcGxldGlvbmAsIHRvIGZhY2lsaXRhdGUgdGhlIGludGVyYWN0aW9uIHdpdGggdGhlIExMTXMgYW5kIGdlbmVyYXRlIGNoYXQgY29tcGxldGlvbnMgZm9yIGVhY2ggbmV3cyBhcnRpY2xlIGluIHRoZSBkYXRhc2V0LiBUaGUgZnVuY3Rpb24gdGFrZXMgdGhlIGZvbGxvd2luZyBwYXJhbWV0ZXJzOiANCg0KLSBgY3N2X3BhdGhgOiB0aGUgcGF0aCB0byB0aGUgQ1NWIGZpbGUgY29udGFpbmluZyB0aGUgbmV3cyBhcnRpY2xlczsNCi0gYGNvbHVtbnNfdG9fa2VlcGA6IHRoZSBjb2x1bW5zIHRvIHJldGFpbiBpbiB0aGUgZmluYWwgQ1NWIG91dHB1dDsNCi0gYG1vZGVsc2A6IGEgbGlzdCBvZiBjaGF0IG1vZGVscyB0byBiZSB1c2VkOw0KLSBgY2hhdF9wcm9tcHRfdGVtcGxhdGVgOiB0aGUgcHJvbXB0IHRlbXBsYXRlIGZvciBnZW5lcmF0aW5nIGNoYXQgbWVzc2FnZXM7DQotIGBjb2x1bW5zX2Zvcl9jaGF0X3Byb21wdGA6IHRoZSBjb2x1bW5zIHRvIGJlIHVzZWQgYXMgdGhlIHVzZXIgaW5wdXQgaW4gdGhlIGNoYXQgcHJvbXB0IHRlbXBsYXRlOw0KLSBgbnVtX3JlcGxpY2F0ZXNgOiB0aGUgbnVtYmVyIG9mIHJlcGxpY2F0ZXMgcGVyIG1vZGVsOw0KLSBgdGVtcGA6IHRoZSB0ZW1wZXJhdHVyZSBmb3IgdGhlIGNoYXQgbW9kZWw7DQotIGBtYXhfbnVtX3Rva2Vuc2A6IHRoZSBtYXhpbXVtIG51bWJlciBvZiB0b2tlbnMgZm9yIGNoYXQgY29tcGxldGlvbnM7DQotIGBzYXZlX3RvX2NzdmA6IHdoZXRoZXIgdG8gc2F2ZSByZXN1bHRzIHRvIENTVjsNCi0gYG91dHB1dF9maWxlYDogdGhlIHBhdGggdG8gdGhlIG91dHB1dCBDU1YgZmlsZTsgYW5kIA0KLSBgcmV0cnlfYXR0ZW1wdHNgOiB0aGUgbnVtYmVyIG9mIHJldHJ5IGF0dGVtcHRzIGZvciBBUEkgZXJyb3JzLg0KDQpUaGUgZnVuY3Rpb24gcmVhZHMgdGhlIENTViBmaWxlLCByZXBsaWNhdGVzIHRoZSBkYXRhIGJhc2VkIG9uIHRoZSBudW1iZXIgb2YgbW9kZWxzIGFuZCByZXBsaWNhdGVzLCBzb3J0cyB0aGUgZGF0YSBmcmFtZSwgYW5kIGl0ZXJhdGVzIHRocm91Z2ggZWFjaCByb3cgdG8gZ2VuZXJhdGUgY2hhdCBjb21wbGV0aW9ucy4gSXQgdXNlcyB0aGUgc3BlY2lmaWVkIGNoYXQgbW9kZWwgdG8gZ2VuZXJhdGUgY2hhdCByZXNwb25zZXMsIHdpdGggZXJyb3IgaGFuZGxpbmcgZm9yIEFQSSBlcnJvcnMuIFRoZSBmdW5jdGlvbiBzYXZlcyB0aGUgcmVzdWx0cyB0byBhIENTViBmaWxlIGlmIHJlcXVpcmVkIGFuZCByZXR1cm5zIHRoZSBsYXN0IGNoYXQgcmVzcG9uc2UgY29udGVudCBpZiBgc2F2ZV90b19jc3ZgIGlzIGBGYWxzZWAuDQoNCmBgYHtweXRob24gY3VzdG9tX2Z1biwgZXZhbD1GQUxTRX0NCmltcG9ydCBvcw0KaW1wb3J0IHRpbWUNCg0KaW1wb3J0IHBhbmRhcyBhcyBwZA0KaW1wb3J0IGRhdGV0aW1lIGFzIGR0DQoNCmZyb20gbGFuZ2NoYWluX29wZW5haSBpbXBvcnQgQ2hhdE9wZW5BSQ0KZnJvbSBsYW5nY2hhaW5fYW50aHJvcGljIGltcG9ydCBDaGF0QW50aHJvcGljDQpmcm9tIGxhbmdjaGFpbl9jb2hlcmUgaW1wb3J0IENoYXRDb2hlcmUNCmZyb20gbGFuZ2NoYWluX29sbGFtYSBpbXBvcnQgQ2hhdE9sbGFtYQ0KDQpkZWYgZ2VuZXJhbGl6ZWRfY2hhdF9jb21wbGV0aW9uKA0KICAgIGNzdl9wYXRoLA0KICAgIGNvbHVtbnNfdG9fa2VlcCwNCiAgICBtb2RlbHMsDQogICAgY2hhdF9wcm9tcHRfdGVtcGxhdGUsDQogICAgY29sdW1uc19mb3JfY2hhdF9wcm9tcHQsDQogICAgbnVtX3JlcGxpY2F0ZXMsDQogICAgdGVtcD0wLA0KICAgIG1heF9udW1fdG9rZW5zPTMwMDAsDQogICAgc2F2ZV90b19jc3Y9VHJ1ZSwNCiAgICBvdXRwdXRfZmlsZT0nLi4vcmVzdWx0cy9nZW5lcmFsaXplZF9jbGFzc2lmaWNhdGlvbi5jc3YnLA0KICAgIHJldHJ5X2F0dGVtcHRzPTMNCik6DQogICAgIiIiDQogICAgR2VuZXJhbGl6ZWQgZnVuY3Rpb24gdG8gZ2VuZXJhdGUgY2hhdCBjb21wbGV0aW9ucyBmcm9tIGEgQ1NWIGZpbGUsIHdpdGggZmxleGlibGUgcGFyYW1ldGVycw0KICAgIGZvciB2YXJpb3VzIGNoYXQgbW9kZWxzLCBzb3J0aW5nIG9yZGVyLCBhbmQgZXJyb3IgaGFuZGxpbmcuDQogICAgDQogICAgUGFyYW1ldGVyczoNCiAgICAgICAgY3N2X3BhdGggKHN0cik6IFBhdGggdG8gdGhlIENTViBmaWxlIHRvIHJlYWQgZGF0YSBmcm9tLg0KICAgICAgICBjb2x1bW5zX3RvX2tlZXAgKGxpc3QpOiBDb2x1bW5zIHRvIHJldGFpbiBpbiB0aGUgZmluYWwgQ1NWIG91dHB1dC4NCiAgICAgICAgbW9kZWxzIChsaXN0KTogTGlzdCBvZiBjaGF0IG1vZGVscyB0byBiZSB1c2VkLg0KICAgICAgICBjaGF0X3Byb21wdF90ZW1wbGF0ZSAoc3RyKTogVGhlIHByb21wdCB0ZW1wbGF0ZSBmb3IgZ2VuZXJhdGluZyBjaGF0IG1lc3NhZ2VzLg0KICAgICAgICBjb2x1bW5zX2Zvcl9jaGF0X3Byb21wdCAobGlzdCk6IENvbHVtbnMgdG8gYmUgdXNlZCBhcyB0aGUgdXNlciBpbnB1dCBpbiB0aGUgY2hhdCBwcm9tcHQgdGVtcGxhdGUuDQogICAgICAgIG51bV9yZXBsaWNhdGVzIChpbnQpOiBOdW1iZXIgb2YgcmVwbGljYXRlcyBwZXIgbW9kZWwuDQogICAgICAgIHRlbXAgKGZsb2F0KTogVGVtcGVyYXR1cmUgZm9yIHRoZSBjaGF0IG1vZGVsLg0KICAgICAgICBtYXhfbnVtX3Rva2VucyAoaW50KTogTWF4aW11bSBudW1iZXIgb2YgdG9rZW5zIGZvciBjaGF0IGNvbXBsZXRpb25zLg0KICAgICAgICBzYXZlX3RvX2NzdiAoYm9vbCk6IFdoZXRoZXIgdG8gc2F2ZSByZXN1bHRzIHRvIENTVi4NCiAgICAgICAgb3V0cHV0X2ZpbGUgKHN0cik6IFBhdGggdG8gdGhlIG91dHB1dCBDU1YgZmlsZS4NCiAgICAgICAgcmV0cnlfYXR0ZW1wdHMgKGludCk6IE51bWJlciBvZiByZXRyeSBhdHRlbXB0cyBmb3IgQVBJIGVycm9ycy4NCiAgICANCiAgICBSZXR1cm5zOg0KICAgICAgICBOb25lIG9yIHRoZSBsYXN0IGNoYXQgcmVzcG9uc2UgY29udGVudCBpZiBzYXZlX3RvX2NzdiBpcyBGYWxzZS4NCiAgICAiIiINCiAgICAjIHJlYWQgdGhlIENTViBmaWxlDQogICAgZGYgPSBwZC5yZWFkX2Nzdihjc3ZfcGF0aCkNCiAgICBudW1fcm93cyA9IGRmLnNoYXBlWzBdDQogICAgdG90YWxfcmVwZWF0cyA9IGxlbihtb2RlbHMpICogbnVtX3JlcGxpY2F0ZXMNCg0KICAgICMgYWRkIGFuIGluZGV4IGNvbHVtbiBhcyAnYXJ0aWNsZV9udW0nDQogICAgZGZbJ2FydGljbGVfbnVtJ10gPSBkZi5pbmRleA0KDQogICAgIyByZXBsaWNhdGUgdGhlIGRhdGEgZnJhbWUgYmFzZWQgb24gdGhlIHRvdGFsIHJlcGVhdHMNCiAgICBleHBhbmRlZF9kZiA9IHBkLmNvbmNhdChbZGZdICogdG90YWxfcmVwZWF0cywgaWdub3JlX2luZGV4PVRydWUpDQoNCiAgICAjIGdlbmVyYXRlIG1vZGVsIGFuZCByZXBsaWNhdGUgY29sdW1ucw0KICAgIG1vZGVsX2NvbHVtbiA9IFttb2RlbCBmb3IgbW9kZWwgaW4gbW9kZWxzIGZvciBfIGluIHJhbmdlKG51bV9yZXBsaWNhdGVzICogbnVtX3Jvd3MpXQ0KICAgIHJlcGxpY2F0ZV9jb2x1bW4gPSBbaSArIDEgZm9yIF8gaW4gcmFuZ2UobGVuKG1vZGVscykpIGZvciBpIGluIHJhbmdlKG51bV9yZXBsaWNhdGVzKSBmb3IgXyBpbiByYW5nZShudW1fcm93cyldDQoNCiAgICAjIGFkZCB0aGUgbW9kZWwgYW5kIHJlcGxpY2F0ZSBjb2x1bW5zIHRvIHRoZSBkYXRhIGZyYW1lDQogICAgZXhwYW5kZWRfZGZbJ3JlcGxpY2F0ZSddID0gcmVwbGljYXRlX2NvbHVtbg0KICAgIGV4cGFuZGVkX2RmWydjaGF0X21vZGVsJ10gPSBtb2RlbF9jb2x1bW4NCg0KICAgICMgc29ydCB0aGUgZGF0YSBmcmFtZSBieSAnY2hhdF9tb2RlbCcsICdhcnRpY2xlX251bScgYW5kICdyZXBsaWNhdGUnDQogICAgZXhwYW5kZWRfZGYgPSBleHBhbmRlZF9kZi5zb3J0X3ZhbHVlcyhieT1bJ2NoYXRfbW9kZWwnLCdhcnRpY2xlX251bScsICdyZXBsaWNhdGUnXSkucmVzZXRfaW5kZXgoZHJvcD1UcnVlKQ0KDQogICAgIyBpdGVyYXRlIHRocm91Z2ggZWFjaCByb3cgYW5kIGdlbmVyYXRlIGNoYXQgY29tcGxldGlvbnMNCiAgICBmb3IgaW5kZXggaW4gcmFuZ2UoZXhwYW5kZWRfZGYuc2hhcGVbMF0pOg0KICAgICAgICBwcm9tcHRfZGF0YSA9IHtjb2w6IGV4cGFuZGVkX2RmLmxvY1tpbmRleCwgY29sXSBmb3IgY29sIGluIGNvbHVtbnNfZm9yX2NoYXRfcHJvbXB0fQ0KICAgICAgICBtZXNzYWdlcyA9IGNoYXRfcHJvbXB0X3RlbXBsYXRlLmZvcm1hdF9tZXNzYWdlcygqKnByb21wdF9kYXRhKQ0KDQogICAgICAgICMgZXh0cmFjdCBtb2RlbCBuYW1lIGFuZCBhc3NpZ24gdGhlIGNvcnJlY3QgY2hhdCBtb2RlbA0KICAgICAgICBtb2RlbCA9IGV4cGFuZGVkX2RmLmxvY1tpbmRleCwgJ2NoYXRfbW9kZWwnXQ0KICAgICAgICBjaGF0X21vZGVsID0gTm9uZQ0KICAgICAgICBpZiBtb2RlbCA9PSAnZ3B0LTRvLTIwMjQtMTEtMjAnOg0KICAgICAgICAgICAgY2hhdF9tb2RlbCA9IENoYXRPcGVuQUkobW9kZWw9ImdwdC00by0yMDI0LTExLTIwIiwgdGVtcGVyYXR1cmU9dGVtcCwgbWF4X3Rva2Vucz1tYXhfbnVtX3Rva2VucykNCiAgICAgICAgZWxpZiBtb2RlbCA9PSAnZ3B0LTRvLW1pbmktMjAyNC0wNy0xOCc6DQogICAgICAgICAgICBjaGF0X21vZGVsID0gQ2hhdE9wZW5BSShtb2RlbD0iZ3B0LTRvLW1pbmktMjAyNC0wNy0xOCIsIHRlbXBlcmF0dXJlPXRlbXAsIG1heF90b2tlbnM9bWF4X251bV90b2tlbnMpDQogICAgICAgIA0KICAgICAgICBlbGlmIG1vZGVsID09ICJjbGF1ZGUtMy03LXNvbm5ldC0yMDI1MDIxOSI6DQogICAgICAgICAgICBjaGF0X21vZGVsID0gQ2hhdEFudGhyb3BpYyhtb2RlbD0iY2xhdWRlLTMtNy1zb25uZXQtMjAyNTAyMTkiLCB0ZW1wZXJhdHVyZT10ZW1wLCBtYXhfdG9rZW5zPW1heF9udW1fdG9rZW5zKQ0KICAgICAgICBlbGlmIG1vZGVsID09ICJjbGF1ZGUtMy01LWhhaWt1LTIwMjQxMDIyIjoNCiAgICAgICAgICAgIGNoYXRfbW9kZWwgPSBDaGF0QW50aHJvcGljKG1vZGVsPSJjbGF1ZGUtMy01LWhhaWt1LTIwMjQxMDIyIiwgdGVtcGVyYXR1cmU9dGVtcCwgbWF4X3Rva2Vucz1tYXhfbnVtX3Rva2VucykNCiAgICAgICAgDQogICAgICAgIGVsaWYgbW9kZWwgPT0gImNvbW1hbmQtci1wbHVzLTA4LTIwMjQiOg0KICAgICAgICAgICAgY2hhdF9tb2RlbCA9IENoYXRDb2hlcmUobW9kZWw9ImNvbW1hbmQtci1wbHVzLTA4LTIwMjQiLCB0ZW1wZXJhdHVyZT10ZW1wLCBtYXhfdG9rZW5zPW1heF9udW1fdG9rZW5zKQ0KICAgICAgICBlbGlmIG1vZGVsID09ICJjb21tYW5kLXI3YiI6DQogICAgICAgICAgICBjaGF0X21vZGVsID0gQ2hhdE9sbGFtYShtb2RlbD0iY29tbWFuZC1yN2IiLCB0ZW1wZXJhdHVyZT10ZW1wLCBtYXhfdG9rZW5zPW1heF9udW1fdG9rZW5zKQ0KICAgICAgICANCiAgICAgICAgZWxpZiBtb2RlbCA9PSAnZGVlcHNlZWstcjE6N0InOg0KICAgICAgICAgICAgY2hhdF9tb2RlbCA9IENoYXRPbGxhbWEobW9kZWw9ImRlZXBzZWVrLXIxOjdCIiwgdGVtcGVyYXR1cmU9dGVtcCwgbWF4X3Rva2Vucz1tYXhfbnVtX3Rva2VucykNCiAgICAgICAgZWxpZiBtb2RlbCA9PSAnZGVlcHNlZWstcjE6MS41Qic6DQogICAgICAgICAgICBjaGF0X21vZGVsID0gQ2hhdE9sbGFtYShtb2RlbD0iZGVlcHNlZWstcjE6MS41QiIsIHRlbXBlcmF0dXJlPXRlbXAsIG1heF90b2tlbnM9bWF4X251bV90b2tlbnMpDQogICAgICAgICAgICANCiAgICAgICAgZWxpZiBtb2RlbCA9PSAnZXhhb25lLWRlZXA6Ny44Yic6IA0KICAgICAgICAgICAgY2hhdF9tb2RlbCA9IENoYXRPbGxhbWEobW9kZWw9ImV4YW9uZS1kZWVwOjcuOGIiLCB0ZW1wZXJhdHVyZT10ZW1wLCBtYXhfdG9rZW5zPW1heF9udW1fdG9rZW5zKQ0KICAgICAgICANCiAgICAgICAgZWxpZiBtb2RlbCA9PSAnZXhhb25lLWRlZXA6Mi40Yic6ICANCiAgICAgICAgICAgIGNoYXRfbW9kZWwgPSBDaGF0T2xsYW1hKG1vZGVsPSJleGFvbmUtZGVlcDoyLjRiIiwgdGVtcGVyYXR1cmU9dGVtcCwgbWF4X3Rva2Vucz1tYXhfbnVtX3Rva2VucykNCiAgICAgICAgDQogICAgICAgIGVsaWYgbW9kZWwgPT0gJ2dlbW1hMzoyN0InOg0KICAgICAgICAgICAgY2hhdF9tb2RlbCA9IENoYXRPbGxhbWEobW9kZWw9ImdlbW1hMzoyN0IiLCB0ZW1wZXJhdHVyZT10ZW1wLCBtYXhfdG9rZW5zPW1heF9udW1fdG9rZW5zKQ0KICAgICAgICBlbGlmIG1vZGVsID09ICdnZW1tYTM6MUInOg0KICAgICAgICAgICAgY2hhdF9tb2RlbCA9IENoYXRPbGxhbWEobW9kZWw9ImdlbW1hMzoxQiIsIHRlbXBlcmF0dXJlPXRlbXAsIG1heF90b2tlbnM9bWF4X251bV90b2tlbnMpDQogICAgICAgIA0KICAgICAgICBlbGlmIG1vZGVsID09ICdsbGFtYTMuMjozQic6DQogICAgICAgICAgICBjaGF0X21vZGVsID0gQ2hhdE9sbGFtYShtb2RlbD0ibGxhbWEzLjI6M0IiLCB0ZW1wZXJhdHVyZT10ZW1wLCBtYXhfdG9rZW5zPW1heF9udW1fdG9rZW5zKQ0KICAgICAgICBlbGlmIG1vZGVsID09ICdsbGFtYTMuMjoxQic6DQogICAgICAgICAgICBjaGF0X21vZGVsID0gQ2hhdE9sbGFtYShtb2RlbD0ibGxhbWEzLjI6MUIiLCB0ZW1wZXJhdHVyZT10ZW1wLCBtYXhfdG9rZW5zPW1heF9udW1fdG9rZW5zKQ0KICAgICAgICANCiAgICAgICAgZWxpZiBtb2RlbCA9PSAncGhpNDpsYXRlc3QnOg0KICAgICAgICAgICAgY2hhdF9tb2RlbCA9IENoYXRPbGxhbWEobW9kZWw9InBoaTQ6bGF0ZXN0IiwgdGVtcGVyYXR1cmU9dGVtcCwgbWF4X3Rva2Vucz1tYXhfbnVtX3Rva2VucykNCiAgICAgICAgZWxpZiBtb2RlbCA9PSAncGhpNC1taW5pJzoNCiAgICAgICAgICAgIGNoYXRfbW9kZWwgPSBDaGF0T2xsYW1hKG1vZGVsPSJwaGk0LW1pbmkiLCB0ZW1wZXJhdHVyZT10ZW1wLCBtYXhfdG9rZW5zPW1heF9udW1fdG9rZW5zKQ0KICAgICAgICANCiAgICAgICAgZWxzZToNCiAgICAgICAgICAgIHByaW50KGYiTW9kZWwge21vZGVsfSBpcyBub3Qgc3VwcG9ydGVkLiBTa2lwcGluZyB0aGlzIHJvdy4iKQ0KICAgICAgICAgICAgY29udGludWUNCg0KICAgICAgICAjIGF0dGVtcHQgdG8gZ2VuZXJhdGUgY2hhdCByZXNwb25zZSB3aXRoIHJldHJpZXMgZm9yIGVycm9yIGhhbmRsaW5nDQogICAgICAgIGNoYXRfcmVzcG9uc2VfY29udGVudCA9ICItLSINCiAgICAgICAgY2hhdF9yZXNwb25zZV9pZCA9IE5vbmUNCiAgICAgICAgZm9yIGF0dGVtcHQgaW4gcmFuZ2UocmV0cnlfYXR0ZW1wdHMpOg0KICAgICAgICAgICAgdHJ5Og0KICAgICAgICAgICAgICAgIGNoYXRfcmVzcG9uc2UgPSBjaGF0X21vZGVsLmludm9rZShtZXNzYWdlcykNCiAgICAgICAgICAgICAgICBjaGF0X3Jlc3BvbnNlX2NvbnRlbnQgPSBjaGF0X3Jlc3BvbnNlLmNvbnRlbnQNCiAgICAgICAgICAgICAgICBjaGF0X3Jlc3BvbnNlX2lkID0gY2hhdF9yZXNwb25zZS5pZA0KICAgICAgICAgICAgICAgIGJyZWFrDQogICAgICAgICAgICBleGNlcHQgRXhjZXB0aW9uIGFzIGU6DQogICAgICAgICAgICAgICAgZXJyb3JfbWVzc2FnZSA9IHN0cihlKQ0KICAgICAgICAgICAgICAgIGlmIGF0dGVtcHQgPT0gcmV0cnlfYXR0ZW1wdHMgLSAxOg0KICAgICAgICAgICAgICAgICAgICBwcmludChmIkZhaWxlZCBhZnRlciB7cmV0cnlfYXR0ZW1wdHN9IGF0dGVtcHRzIGZvciBtb2RlbCB7bW9kZWx9IG9uIGluZGV4IHtpbmRleH0uIEVycm9yOiB7ZXJyb3JfbWVzc2FnZX0iKQ0KICAgICAgICAgICAgICAgIGVsc2U6DQogICAgICAgICAgICAgICAgICAgIHByaW50KGYiQXR0ZW1wdCB7YXR0ZW1wdCArIDF9IGZhaWxlZCBmb3IgbW9kZWwge21vZGVsfSBvbiBpbmRleCB7aW5kZXh9LiBSZXRyeWluZyBpbiAzMCBzZWNvbmRzLi4uIikNCiAgICAgICAgICAgICAgICAgICAgdGltZS5zbGVlcCgzMCkNCg0KICAgICAgICAjIGNyZWF0ZSB0aGUgcm93IHdpdGggdGhlIGRlc2lyZWQgY29sdW1ucw0KICAgICAgICBkYXRhX3JvdyA9IHBkLkRhdGFGcmFtZSh7DQogICAgICAgICAgICAqKntjb2w6IFtleHBhbmRlZF9kZi5sb2NbaW5kZXgsIGNvbF1dIGZvciBjb2wgaW4gY29sdW1uc190b19rZWVwfSwNCiAgICAgICAgICAgICdjaGF0X21vZGVsJzogW21vZGVsXSwNCiAgICAgICAgICAgICdjaGF0X2RhdGUnOiBbZHQuZGF0ZXRpbWUubm93KCkuc3RyZnRpbWUoJyVZLSVtLSVkICVIOiVNOiVTJyldLA0KICAgICAgICAgICAgJ2NoYXRfcmVwbGljYXRlJzogW2V4cGFuZGVkX2RmLmxvY1tpbmRleCwgJ3JlcGxpY2F0ZSddXSwNCiAgICAgICAgICAgICdjaGF0X3Jlc3BvbnNlJzogW2NoYXRfcmVzcG9uc2VfY29udGVudF0sDQogICAgICAgICAgICAnY2hhdF9yZXNwb25zZV9pZCc6IFtjaGF0X3Jlc3BvbnNlX2lkXQ0KICAgICAgICB9KQ0KDQogICAgICAgICMgc2F2ZSB0byBDU1YgaWYgcmVxdWlyZWQNCiAgICAgICAgaWYgbm90IHNhdmVfdG9fY3N2Og0KICAgICAgICAgICAgY29udGludWUNCiAgICAgICAgZWxpZiBub3Qgb3MucGF0aC5leGlzdHMob3V0cHV0X2ZpbGUpOg0KICAgICAgICAgICAgZGF0YV9yb3cudG9fY3N2KG91dHB1dF9maWxlLCBpbmRleD1GYWxzZSkNCiAgICAgICAgZWxzZToNCiAgICAgICAgICAgIGV4aXN0aW5nX2RmID0gcGQucmVhZF9jc3Yob3V0cHV0X2ZpbGUpDQogICAgICAgICAgICB1cGRhdGVkX2RmID0gcGQuY29uY2F0KFtleGlzdGluZ19kZiwgZGF0YV9yb3ddLCBpZ25vcmVfaW5kZXg9VHJ1ZSkNCiAgICAgICAgICAgIHVwZGF0ZWRfZGYudG9fY3N2KG91dHB1dF9maWxlLCBpbmRleD1GYWxzZSkNCiAgICANCiAgICBpZiBub3Qgc2F2ZV90b19jc3Y6DQogICAgICAgIHJldHVybiBjaGF0X3Jlc3BvbnNlX2NvbnRlbnQNCg0KYGBgDQoNCg0KIyMgQSBDdXN0b20gRnVuY3Rpb24gZm9yIEV4dHJhY3RpbmcgdGhlIExMTSBMYWJlbHMgeyNjdXN0b21fZXh0cmFjdF9sYWJlbHN9DQoNCldlIGNyZWF0ZWQgYSBjdXN0b20gZnVuY3Rpb24sIGBleHRyYWN0X2NsYXNzaWZpY2F0aW9uYCwgdG8gZXh0cmFjdCB0aGUgY2xhc3NpZmljYXRpb24gbGFiZWxzIGZyb20gdGhlIGNoYXQgY29tcGxldGlvbnMgZ2VuZXJhdGVkIGJ5IHRoZSBMTE1zLiBUaGUgZnVuY3Rpb24gdGFrZXMgdGhlIGNoYXQgcmVzcG9uc2UgY29udGVudCBhbmQgYSBsaXN0IG9mIHZhbGlkIHdvcmRzIGFzIGlucHV0LiBJdCBleHRyYWN0cyB0aGUgY2xhc3NpZmljYXRpb24gbGFiZWwgZnJvbSB0aGUgY2hhdCByZXNwb25zZSBjb250ZW50IGJhc2VkIG9uIHRoZSB2YWxpZCB3b3JkcyBwcm92aWRlZC4gVGhlIGZ1bmN0aW9uIHJldHVybnMgdGhlIGV4dHJhY3RlZCBjbGFzc2lmaWNhdGlvbiBsYWJlbCBpZiBpdCBtYXRjaGVzIGFueSBvZiB0aGUgdmFsaWQgd29yZHM7IG90aGVyd2lzZSwgaXQgcmV0dXJucyBgTkFgLg0KDQpgYGB7ciBjdXN0b21fZXh0cmFjdF9sYWJlbHN9DQpleHRyYWN0X2NsYXNzaWZpY2F0aW9uIDwtIGZ1bmN0aW9uKHRleHQsIHZhbGlkX3dvcmRzKSB7DQogICMgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KICAjIEhlbHBlcjogZHJvcCByb3dzIHdpdGggdW53YW50ZWQgd29yZHMNCiAgIyAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQogIA0KICBkcm9wX3JvdyA9IGZ1bmN0aW9uKHJvdykgeyANCiAgICBhbGwoaXMubmEocm93KSB8IHJvdyA9PSAiY2xhc3NpZmljYXRpb24uXG5cbnRlbXAiKQ0KICB9DQogIA0KICAjIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCiAgIyBIZWxwZXI6IFByZS1jbGVhbiB0ZXh0MQ0KICAjIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCiAgY2xlYW5fdGV4dDEgPC0gZnVuY3Rpb24odGV4dCkgew0KICAgIHBhdHRlcm5zIDwtIGMoDQogICAgICAiaGVyZSAiLCAiIiwgDQogICAgICAiY2xhc3NpZmljYXRpb247IiwgIiIsIA0KICAgICAgImRlZmluaXRpdmUgY2xhc3NpZmljYXRpb24sIiwgIiIsIA0KICAgICAgIlRoZSBjbGFzc2lmaWNhdGlvbiBpcyBiYXNlZCBvbiIsICIiLCANCiAgICAgICJjbGFzc2lmaWNhdGlvbiBiYXNlZCBvbiIsICIiLCANCiAgICAgICJjbGFzc2lmaWNhdGlvbiBiYXNlZCAiLCAiIiwgDQogICAgICAiY2xhc3NpZmljYXRpb24gcmVxdWlyZXMiLCAiaXQgcmVxdWlyZXMiLCANCiAgICAgICJjbGFzc2lmaWNhdGlvbiB3aXRob3V0IiwgIiIsIA0KICAgICAgIml0IGlzIGltcG9zc2libGUiLCAiY2xhc3NpZmljYXRpb246IEltcG9zc2libGUiLCANCiAgICAgICJjaGFsbGVuZ2luZyBjbGFzc2lmaWNhdGlvbiIsICIiLCANCiAgICAgICJ0aGUgY2xhc3NpZmljYXRpb24gY2hhbGxlbmdpbmciLCAiaXQgY2hhbGxlbmdpbmciLCANCiAgICAgICJcbi0tLVxuIiwgIiAiLCANCiAgICAgICJcXCoiLCAiIiwgDQogICAgICAnXFwiJywgIiIsIA0KICAgICAgImNhbm5vdCBkZWZpbml0aXZlbHkgY2xhc3NpZnkgdGhpcyBuZXdzIGFzIGVpdGhlciBQb3NpdGl2ZSBvciBOZWdhdGl2ZSIsICIiLCANCiAgICAgICJlaXRoZXIgUG9zaXRpdmUgb3IgTmVnYXRpdmUiLCAiIiwgDQogICAgICAiYXM6IiwgImFzIiwgDQogICAgICAiXG48L2FuYWx5c2lzPjogXG48Y2xhc3NpZmljYXRpb24+OiIsICIgY2xhc3NpZmljYXRpb246IiwgDQogICAgICAibWFraW5nIGl0IGNoYWxsZW5naW5nIHRvIiwgIiIsIA0KICAgICAgImNsYXNzaWZpY2F0aW9uLjwvY2xhc3NpZmljYXRpb24+IiwgIiIsIA0KICAgICAgIlxuXG5DbGFzc2lmaWNhdGlvbjpcbi0iLCAiY2xhc3NpZmljYXRpb246IiwgDQogICAgICAiY2xhc3NpZmljYXRpb24uXG5cbjxjbGFzc2lmaWNhdGlvbj46IiwgImNsYXNzaWZpY2F0aW9uOiIsIA0KICAgICAgImFydGljbGUiLCAibmV3cyIsIA0KICAgICAgIjwvYW5hbHlzaXM+IiwgIjxhbmFseXNpcz4iLCANCiAgICAgICJjbGFzc2lmaWNhdGlvblxcLiAiLCAiIiwgDQogICAgICAiY2xhc3NpZmljYXRpb24gYmVjYXVzZXxjbGFzc2lmaWNhdGlvbiBmb3IiLCAiIiwgDQogICAgICAicG9zaXRpdmVseSIsICJwb3NpdGl2ZSIsIA0KICAgICAgIm5lZ2F0aXZlbHkiLCAibmVnYXRpdmUiLCANCiAgICAgICJsZWFuaW5nIHNsaWdodGx5IHRvd2FyZHMiLCAiY2xhc3NpZmljYXRpb246IiwgDQogICAgICAidGhlIGltcGFjdCBpcyBsaWtlbHkgdG8gYmUiLCAiY2xhc3NpZmljYXRpb246Ig0KICAgICkNCiAgICANCiAgICBmb3IgKGkgaW4gc2VxKDEsIGxlbmd0aChwYXR0ZXJucyksIGJ5ID0gMikpIHsNCiAgICAgIHRleHQgPC0gZ3N1YihwYXR0ZXJuc1tpXSwgcGF0dGVybnNbaSArIDFdLCB0ZXh0KQ0KICAgIH0NCiAgICANCiAgICByZXR1cm4odGV4dCkNCiAgfQ0KICANCiAgIyAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQogICMgSGVscGVyOiBOb3JtYWxpemUgc2VudGltZW50IHBhdHRlcm4xIHRvIHN0YW5kYXJkIGZvcm1hdA0KICAjIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCiAgbm9ybWFsaXplX3NlbnRpbWVudF9waHJhc2VzMSA8LSBmdW5jdGlvbih0ZXh0KSB7DQogICAgc2VudGltZW50X3BhdHRlcm5zIDwtIGMoDQogICAgICAiXFxuXFxzKjxhbmFseXNpcz46IiwNCiAgICAgICI8Y2xhc3NpZmljYXRpb24+OiguKj8paXMgbGlrZWx5IHRvIGJlIiwNCiAgICAgICJsaWtlbHkgYmUiLA0KICAgICAgImlzIGxpa2VseSB0byBiZSIsDQogICAgICAiaXMgZXhwZWN0ZWQgdG8gaGF2ZSBhIiwNCiAgICAgICJDbGFzc2lmaWNhdGlvbjooLio/KWlzIGV4cGVjdGVkIHRvIiwNCiAgICAgICJjbGFzc2lmaWNhdGlvbiBsZWFucyB0b3dhcmRzIiwNCiAgICAgICJpcyBsaWtlbHkiLA0KICAgICAgImlzIGxpa2VseSB0byBoYXZlIGEiLA0KICAgICAgImxlYWRpbmcgdG8gYSIsDQogICAgICAibGlrZWx5IGxlYWRzIHRvIGEiLA0KICAgICAgInN1Z2dlc3QgYSBwb3RlbnRpYWwiLA0KICAgICAgIk92ZXJhbGwsIHRoZSBpbW1lZGlhdGUiDQogICAgKQ0KICAgIA0KICAgIHNlbnRpbWVudF90eXBlcyA8LSAiKHBvc2l0aXZlfG5lZ2F0aXZlfG5ldXRyYWx8bWl4ZWQpIg0KICAgIA0KICAgIGZvciAocGF0dGVybiBpbiBzZW50aW1lbnRfcGF0dGVybnMpIHsNCiAgICAgIGZ1bGxfcGF0dGVybiA8LSByZWdleChwYXN0ZTAocGF0dGVybiwgIlxccyoiLCBzZW50aW1lbnRfdHlwZXMsICJcXHMqXFxuPyIpLCBpZ25vcmVfY2FzZSA9IFRSVUUpDQogICAgICB0ZXh0IDwtIHN0cl9yZXBsYWNlX2FsbCgNCiAgICAgICAgdGV4dCwNCiAgICAgICAgZnVsbF9wYXR0ZXJuLA0KICAgICAgICBmdW5jdGlvbihtKSB7DQogICAgICAgICAgc2VudGltZW50IDwtIHN0cl9leHRyYWN0KG0sICIoP2kpcG9zaXRpdmV8bmVnYXRpdmV8bmV1dHJhbHxtaXhlZCIpDQogICAgICAgICAgcGFzdGUwKCI8Y2xhc3NpZmljYXRpb24+OiAiLCBzdHJfdG9fdGl0bGUoc2VudGltZW50KSwgIiAiKQ0KICAgICAgICB9DQogICAgICApDQogICAgfQ0KICAgIA0KICAgIHJldHVybih0ZXh0KQ0KICB9DQogICMgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KICAjIFByZS1jbGVhbiB0ZXh0Mg0KICAjIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCiAgY2xlYW5fdGV4dDIgPC0gZnVuY3Rpb24odGV4dCkgew0KICAgIHBhdHRlcm5zIDwtIGMoDQogICAgICAiXG57MSwyfSg8Y2xhc3NpZmljYXRpb24+KSIsICJcXDEiLCANCiAgICAgICJjbGFzc2lmaWNhdGlvbjpcblxuPGFuYWx5c2lzPiIsICIiLCANCiAgICAgICJ0aGUgY2xhc3NpZmljYXRpb24gYmVjb21lcyBjaGFsbGVuZ2luZyIsICJpdCBiZWNvbWVzIGNoYWxsZW5naW5nIiwgDQogICAgICAic3RyaWN0IGNsYXNzaWZpY2F0aW9uIGd1aWRlbGluZXNcXC4iLCAiIiwgDQogICAgICAiY2xhc3NpZmljYXRpb24gZ3VpZGVsaW5lczp8Y2xhc3NpZmljYXRpb24gZ3VpZGVsaW5lcyIsICJjbGFzc2lmaWNhdGlvbjoiLCANCiAgICAgICJcblxuQ2xhc3NpZmljYXRpb246XG5UaGUgbmV3cyBoYXMgYSIsICIgY2xhc3NpZmljYXRpb246IiwgDQogICAgICAiY2xhc3NpZmljYXRpb24gZHVlIHRvIiwgIiIsIA0KICAgICAgImNhdGVnb3JpemUgdGhpcyBuZXdzIGFzIGhhdmluZyBlaXRoZXIiLCAidG8iLCANCiAgICAgICJjYXRlZ29yaXplIHRoaXMgbmV3cyBhc3xjbGFzc2lmeSB0aGlzIG5ld3MgYXN8Y2xhc3NpZnkgdGhpcyBhc3x0aGUgY2xhc3NpZmljYXRpb24gc2hvdWxkIGJlfHN0aWxsIGNsYXNzaWZpZWQgaXQgYXN8SSB3b3VsZCBjYXRlZ29yaXplIHRoaXMgYXN8Y2xhc3NpZmllZCBhc3xpcyBjYXRlZ29yaXplZCBhc3xtYWtpbmcgaXR8bWFraW5nIGl0IGEiLCAiY2xhc3NpZmljYXRpb246IiwgDQogICAgICAiXG48L2NsYXNzaWZpY2F0aW9uPiIsICIiLCANCiAgICAgICJJIGNsYXNzaWZ5IHRoZSBpbXBhY3Qgb2YgdGhpcyBuZXdzIiwgIkkgd291bGQgY2xhc3NpZnkgdGhlIGltcGFjdCBvZiB0aGlzIG5ld3MiLCANCiAgICAgICJJIHdvdWxkIGNsYXNzaWZ5IHRoZSBuZXdzIiwgIkkgd291bGQgY2xhc3NpZnkgdGhlIGltcGFjdCBvZiB0aGlzIG5ld3MiDQogICAgKQ0KICAgIA0KICAgIGZvciAoaSBpbiBzZXEoMSwgbGVuZ3RoKHBhdHRlcm5zKSwgYnkgPSAyKSkgew0KICAgICAgdGV4dCA8LSBnc3ViKHBhdHRlcm5zW2ldLCBwYXR0ZXJuc1tpICsgMV0sIHRleHQpDQogICAgfQ0KICAgIA0KICAgIHJldHVybih0ZXh0KQ0KICB9DQogIA0KICAjIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCiAgIyBIZWxwZXI6IE1hdGNoIGEgc2VudGVuY2UgdGhhdCBzdGFydHMgd2l0aCB0aGUgY2xhc3NpZmljYXRpb24gcGhyYXNlIGFuZCBlbmRzIHdpdGggYSBsYWJlbA0KICAjIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCiAgZGV0ZWN0X3NlbnRpbWVudF9mcm9tX2NsYXNzaWZpY2F0aW9uX3NlbnRlbmNlIDwtIGZ1bmN0aW9uKHRleHQpIHsNCiAgICBtYXRjaCA8LSBzdHJfbWF0Y2goDQogICAgICB0ZXh0LA0KICAgICAgcmVnZXgoDQogICAgICAgICJJIHdvdWxkIGNsYXNzaWZ5IHRoZSBpbXBhY3QuKj9hc1xccytbXCInXT8ocG9zaXRpdmV8bmVnYXRpdmV8bWl4ZWQpW1wiJ10/Wy4/IV0/IiwNCiAgICAgICAgaWdub3JlX2Nhc2UgPSBUUlVFDQogICAgICApDQogICAgKQ0KICAgIA0KICAgIGlmICghaXMubmEobWF0Y2hbMl0pKSB7DQogICAgICBvdXRwdXQgPC0gcGFzdGUwKCI8Y2xhc3NpZmljYXRpb24+OiAiLCBzdHJfdG9fdGl0bGUobWF0Y2hbMl0pLCAiICIpDQogICAgICByZXR1cm4ob3V0cHV0KSAgIyBDYXBpdGFsaXplDQogICAgfSBlbHNlIHsNCiAgICAgIHJldHVybih0ZXh0KQ0KICAgIH0NCiAgfQ0KICAjIEhlbHBlcjogTm9ybWFsaXplIHNlbnRpbWVudCBwYXR0ZXJuMiB0byBzdGFuZGFyZCBmb3JtYXQNCiAgIyAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQogIG5vcm1hbGl6ZV9zZW50aW1lbnRfcGhyYXNlczIgPC0gZnVuY3Rpb24odGV4dCkgew0KICAgIHNlbnRpbWVudF9wYXR0ZXJucyA8LSBjKA0KICAgICAgIkkgd291bGQgY2xhc3NpZnkgdGhpcyBuZXdzIG5ld3MgYXMiLCANCiAgICAgICJ0aGUgY2xhc3NpZmljYXRpb24gaXMiLA0KICAgICAgInRoZSB0b25lIG9mIHRoZSBuZXdzIGlzIiwNCiAgICAgICJ0aGUgbmV3cyBwcmVzZW50cyBhIg0KICAgICkNCiAgICANCiAgICBzZW50aW1lbnRfdHlwZXMgPC0gIihwb3NpdGl2ZXxuZWdhdGl2ZXxuZXV0cmFsfG1peGVkKSINCiAgICANCiAgICBmb3IgKHBhdHRlcm4gaW4gc2VudGltZW50X3BhdHRlcm5zKSB7DQogICAgICBmdWxsX3BhdHRlcm4gPC0gcmVnZXgocGFzdGUwKHBhdHRlcm4sICJcXHMqIiwgc2VudGltZW50X3R5cGVzLCAiXFxzKlxcbj8iKSwgaWdub3JlX2Nhc2UgPSBUUlVFKQ0KICAgICAgdGV4dCA8LSBzdHJfcmVwbGFjZV9hbGwoDQogICAgICAgIHRleHQsDQogICAgICAgIGZ1bGxfcGF0dGVybiwNCiAgICAgICAgZnVuY3Rpb24obSkgew0KICAgICAgICAgIHNlbnRpbWVudCA8LSBzdHJfZXh0cmFjdChtLCAiKD9pKXBvc2l0aXZlfG5lZ2F0aXZlfG5ldXRyYWx8bWl4ZWQiKQ0KICAgICAgICAgIHBhc3RlMCgiPGNsYXNzaWZpY2F0aW9uPjogIiwgc3RyX3RvX3RpdGxlKHNlbnRpbWVudCksICIgIikNCiAgICAgICAgfQ0KICAgICAgKQ0KICAgIH0NCiAgICANCiAgICByZXR1cm4odGV4dCkNCiAgfQ0KICANCiAgIyAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQogICMgSGVscGVyOiBEZXRlY3QgIkl0IGlzIGxpa2VseSB0aGF0Li4uIiBzZW50ZW5jZXMNCiAgIyAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQogIGRldGVjdF9saWtlbHlfc2VudGltZW50IDwtIGZ1bmN0aW9uKHRleHQpew0KICAgICMgRXh0cmFjdCBhbGwgc2VudGVuY2VzIHRoYXQgc3RhcnQgd2l0aCAiSXQgaXMgbGlrZWx5IHRoYXQiDQogICAgbWF0Y2hlcyA8LSBzdHJfbWF0Y2hfYWxsKA0KICAgICAgdGV4dCwNCiAgICAgIHJlZ2V4KCJpdCBpcyBsaWtlbHkgdGhhdFteLj8hXSpcXGIocG9zaXRpdmV8bmVnYXRpdmUpXFxiW14uPyFdKlsuPyFdIiwgaWdub3JlX2Nhc2UgPSBUUlVFKQ0KICAgIClbWzFdXQ0KICAgIA0KICAgIGlmIChucm93KG1hdGNoZXMpID4gMCkgew0KICAgICAgb3V0cHV0IDwtIHBhc3RlMCgiPGNsYXNzaWZpY2F0aW9uPjogIiwgc3RyX3RvX3RpdGxlKG1hdGNoZXNbMSwgMl0pLCAiICIpICMgIlBvc2l0aXZlIiBvciAiTmVnYXRpdmUiDQogICAgICByZXR1cm4ob3V0cHV0KQ0KICAgIH0gZWxzZSB7DQogICAgICByZXR1cm4odGV4dCkNCiAgICB9DQogIH0NCiAgDQogICMgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KICAjIFN0ZXAgMTogQ2xlYW4gYW5kIE5vcm1hbGl6ZQ0KICAjIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCiAgdGV4dCA8LSBjbGVhbl90ZXh0MSh0ZXh0KQ0KICB0ZXh0IDwtIG5vcm1hbGl6ZV9zZW50aW1lbnRfcGhyYXNlczEodGV4dCkNCiAgDQogIHRleHQgPC0gZ3N1YigiPC9jbGFzc2lmaWNhdGlvbj46IiwgIjxjbGFzc2lmaWNhdGlvbj46IiwgdGV4dCkNCiAgdGV4dF90ZW1wIDwtIHN0cl9leHRyYWN0KHRleHQsICIoP2kpPGNsYXNzaWZpY2F0aW9uPjpcXHMqKHBvc2l0aXZlfG5lZ2F0aXZlfG5ldXRyYWx8bWl4ZWQpXFxzKiQiKQ0KICBpZiAoIWlzLm5hKHRleHRfdGVtcCkpIHRleHQgPC0gdGV4dF90ZW1wDQogIA0KICB0ZXh0IDwtIGNsZWFuX3RleHQyKHRleHQpDQogIHRleHQgPC0gZGV0ZWN0X3NlbnRpbWVudF9mcm9tX2NsYXNzaWZpY2F0aW9uX3NlbnRlbmNlKHRleHQpDQogIHRleHQgPC0gZGV0ZWN0X2xpa2VseV9zZW50aW1lbnQodGV4dCkNCiAgdGV4dCA8LSBub3JtYWxpemVfc2VudGltZW50X3BocmFzZXMyKHRleHQpDQogIA0KICB0ZXh0ID0gc3RyaW5ncjo6c3RyX3JlcGxhY2UodGV4dCwgIk1peGVkIHRvICIsICIiKQ0KICB0ZXh0ID0gc3RyaW5ncjo6c3RyX3JlcGxhY2VfYWxsKHRleHQsIGMoDQogICAgIlNsaWdodGx5IiA9ICIiLCAjIFJlbW92ZSB0aGUgd29yZCAic2xpZ2h0bHkiDQogICAgImNsZWFybHkiID0gIiIsICAgICAgIyBSZW1vdmUgdGhlIHdvcmQgImNsZWFybHkiDQogICAgJ1xcIicgPSAiIiwgICAgICAgICAgIyBSZW1vdmUgZXNjYXBlZCBxdW90ZXMgKFxcIikNCiAgICAnIicgPSAiIiAgICAgICAgICAgICAjIFJlbW92ZSByZWd1bGFyIHF1b3RlcyAoIikNCiAgKSkNCiAgDQogIGlmIChzdHJfZGV0ZWN0KHRleHQsICI8Y2xhc3NpZmljYXRpb24+OiBQb3NpdGl2ZSIpKSB0ZXh0ID0gIjxjbGFzc2lmaWNhdGlvbj46IFBvc2l0aXZlIg0KICBpZiAoc3RyX2RldGVjdCh0ZXh0LCAiPGNsYXNzaWZpY2F0aW9uPjogTmVnYXRpdmUiKSkgdGV4dCA9ICI8Y2xhc3NpZmljYXRpb24+OiBOZWdhdGl2ZSINCiAgaWYgKHN0cl9kZXRlY3QodGV4dCwgIjxjbGFzc2lmaWNhdGlvbj46IE5ldXRyYWwiKSkgdGV4dCA9ICI8Y2xhc3NpZmljYXRpb24+OiBOZXV0cmFsIg0KICANCiAgIyAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQogICMgU3RlcCAyOiBFeHRyYWN0IGZpbmFsIGNsYXNzaWZpY2F0aW9uDQogICMgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KICAjIGRlZmluZSBhIHJlZ2V4IHBhdHRlcm4gdG8gbWF0Y2g6DQogICMgMS4gIlxuXG5jbGFzc2lmaWNhdGlvbiIgd2l0aCBvciB3aXRob3V0IHNwZWNpYWwgY2hhcmFjdGVycyBvciBjb2xvbnMgYWZ0ZXIgaXQNCiAgIyAyLiAiXG5cbjxjbGFzc2lmaWNhdGlvbj46IiBvciAiXG5cbjxjbGFzc2lmaWNhdGlvbj53b3JkPC9jbGFzc2lmaWNhdGlvbj4iDQogIG1hdGNoZXMgPSBzdHJpbmdyOjpzdHJfbWF0Y2hfYWxsKHRleHQsICIoP2kpKGNsYXNzaWZpY2F0aW9uW15hLXpBLVpdKjo/XFxzKlxcdyt8PGNsYXNzaWZpY2F0aW9uPihcXHcrKTwvY2xhc3NpZmljYXRpb24+KSIpDQogIA0KICAjIGdldCB0aGUgbGFzdCBtYXRjaCBpZiBpdCBleGlzdHMNCiAgaWYgKGxlbmd0aChtYXRjaGVzW1sxXV0pID4gMCkgew0KICAgIGlmIChucm93KG1hdGNoZXNbWzFdXSkgPiAxKSB7ICANCiAgICAgICMgQXBwbHkgdGhlIGZ1bmN0aW9uIHRvIGZpbHRlciBvdXQgdW53YW50ZWQgcm93cw0KICAgICAgbWF0Y2hlc1tbMV1dID0gbWF0Y2hlc1tbMV1dWyFhcHBseShtYXRjaGVzW1sxXV0sIDEsIGRyb3Bfcm93KSwgLCBkcm9wID0gRkFMU0VdDQogICAgfQ0KICAgIGZpcnN0X21hdGNoID0gaGVhZChtYXRjaGVzW1sxXV0sIDEpDQogICAgbGFzdF9tYXRjaCA9IHRhaWwobWF0Y2hlc1tbMV1dLCAxKQ0KICAgIHdvcmQgPSBOQQ0KICAgIHdvcmQxID0gTkENCiAgICB3b3JkMiA9IE5BDQogICAgIyBIYW5kbGUgdGhlIGNhc2UgZm9yICI8Y2xhc3NpZmljYXRpb24+d29yZDwvY2xhc3NpZmljYXRpb24+Ig0KICAgIGlmICghaXMubmEoZmlyc3RfbWF0Y2hbLCAzXSkpIHsNCiAgICAgIHdvcmQgPSBmaXJzdF9tYXRjaFssIDNdICAjIEV4dHJhY3Qgd29yZCBiZXR3ZWVuIDxjbGFzc2lmaWNhdGlvbj4gYW5kIDwvY2xhc3NpZmljYXRpb24+DQogICAgfWVsc2UgaWYgKCFpcy5uYShsYXN0X21hdGNoWywgM10pKSB7DQogICAgICB3b3JkID0gbGFzdF9tYXRjaFssIDNdDQogICAgfWVsc2Ugew0KICAgICAgd29yZDEgPSBzdHJpbmdyOjpzdHJfZXh0cmFjdChmaXJzdF9tYXRjaFssIDFdLCAiXFx3KyQiKSAgIyBPdGhlcndpc2UsIGV4dHJhY3Qgd29yZCBhZnRlciAiY2xhc3NpZmljYXRpb24iDQogICAgICB3b3JkMiA9IHN0cmluZ3I6OnN0cl9leHRyYWN0KGxhc3RfbWF0Y2hbLCAxXSwgIlxcdyskIikNCiAgICB9DQogICAgDQogICAgIyBDaGVjayBpZiB0aGUgZXh0cmFjdGVkIHdvcmQgaXMgaW4gdGhlIHZhbGlkX3dvcmRzIGxpc3QNCiAgICBpZiAodG9sb3dlcih3b3JkKSAlaW4lIHRvbG93ZXIodmFsaWRfd29yZHMpKSB7DQogICAgICByZXR1cm4od29yZCkNCiAgICB9IGVsc2UgaWYgKHRvbG93ZXIod29yZDEpICVpbiUgdG9sb3dlcih2YWxpZF93b3Jkcykpew0KICAgICAgcmV0dXJuKHdvcmQxKQ0KICAgIH0gZWxzZSBpZiAodG9sb3dlcih3b3JkMikgJWluJSB0b2xvd2VyKHZhbGlkX3dvcmRzKSl7DQogICAgICByZXR1cm4od29yZDIpDQogICAgfSBlbHNlIHsNCiAgICAgIHJldHVybihOQSkgICMgUmV0dXJuIE5BIGlmIG5vIG1hdGNoIGlzIGZvdW5kDQogICAgfQ0KICB9IGVsc2Ugew0KICAgIHJldHVybihOQSkgICMgUmV0dXJuIE5BIGlmIG5vIG1hdGNoIGlzIGZvdW5kDQogIH0NCn0NCg0KYGBgDQoNCiMjIEN1c3RvbSBGdW5jdGlvbnMgZm9yIENvbXB1dGluZyB0aGUgUmVsaWFiaWxpdHkgTWV0cmljcyB7I2N1c3RvbV9yZWxpYWJpbGl0eV9tZXRyaWNzfQ0KDQpgYGB7ciBjb21wdXRlX3JlbGlhYmlsaXR5X21ldHJpY3N9DQojIEZ1bmN0aW9uIHRvIGdldCBzdW1tYXJ5IHN0YXRpc3RpY3MgZm9yIG1lYW4gcGVyY2VudCBhZ3JlZW1lbnQNCnBhX3N1bW1hcnkgPSBmdW5jdGlvbihkZiwgcGVyY2VudF92YXJpYWJsZSwgZGlnaXRzPTQpew0KICBsbG1fbW9kZWxzID0gdW5pcXVlKGRmJGNoYXRfbW9kZWwpDQogIGRmX3N1bW1hcnkgPSBkYXRhLmZyYW1lKG1vZGVsID0gbGxtX21vZGVscywgbVBhID0gcmVwKE5BLCBsZW5ndGgobGxtX21vZGVscykpKQ0KICANCiAgZm9yIChpIGluIDE6bGVuZ3RoKGxsbV9tb2RlbHMpKXsNCiAgICBtMSA9IGRmIHw+IA0KICAgICAgZHBseXI6OmZpbHRlcihjaGF0X21vZGVsID09IGxsbV9tb2RlbHNbaV0pIHw+DQogICAgICBkcGx5cjo6c2VsZWN0KHBlcmNlbnRfdmFyaWFibGUpDQogICAgY29sbmFtZXMobTEpIDwtICJ2MSINCiAgICANCiAgICBkZl9zdW1tYXJ5W2ksMl0gPSBwYXN0ZTAoDQogICAgICByb3VuZChtZWFuKG0xJHYxLzEwMCksIGRpZ2l0cyksICIgKCIsIA0KICAgICAgcm91bmQoc2QobTEkdjEvMTAwKSwgZGlnaXRzKSwgICAiKSINCiAgICApDQogIH0NCiAgcmV0dXJuKGRmX3N1bW1hcnkpDQp9DQoNCiMgZnVuY3Rpb24gdG8gY29tcHV0ZSB0aGUgb3RoZXIgcmVsaWFiaWxpdHkgbWV0cmljcw0KIyBzaG91bGQgY29udGFpbiBvbmUgdmFyaWFibGUgY2FsbGVkIGNoYXRfbW9kZWwsDQojIGFuZCB2YXJzIGluZGljYXRlcyB2YXJpYWJsZXMgZm9yIHJhdGVycw0KcmVsaWFiaWxpdHlfY29lZnMgPSBmdW5jdGlvbihkZiwgdmFycyl7DQogIG5vX21vZGVscyA8LSBuX2Rpc3RpbmN0KGRmJGNoYXRfbW9kZWwpDQogIGRmX2NvZWZzID0gZGF0YS5mcmFtZShtYXRyaXgoTkEsIG5jb2w9OSwgbnJvdz01Km5vX21vZGVscykpDQogIA0KICBjb2xuYW1lcyhkZl9jb2VmcykgPSBjKCJtb2RlbCIsICJjb2VmZi5uYW1lIiwgInBhIiwgInBlIiwgImNvZWZmLnZhbCIsICJjb2VmZi5zZSIsICJjb25mLmludCIsICJwLnZhbHVlIiwgIncubmFtZSIpDQogIA0KICBsbG1fbW9kZWxzID0gdW5pcXVlKGRmJGNoYXRfbW9kZWwpDQogIA0KICBkZl9jb2VmcyRtb2RlbCA9IHJlcChsbG1fbW9kZWxzLCBlYWNoPTUpDQogIA0KICBpZiAoaXMuaW50ZWdlcih2YXJzKSkgdmFycyA9IGNvbG5hbWVzKGRmKVt2YXJzXQ0KICANCiAgZGZfcmF0ZXMgPSBkZiB8Pg0KICAgIGRwbHlyOjpzZWxlY3QoZHBseXI6OmFsbF9vZihjKCJjaGF0X21vZGVsIiwgdmFycykpKQ0KICANCiAgZm9yIChpIGluIHNlcSgxLCA1Km5vX21vZGVscywgYnk9NSkpew0KICAgIG0xID0gZGZfcmF0ZXMgfD4NCiAgICAgIGRwbHlyOjpmaWx0ZXIoY2hhdF9tb2RlbCA9PSBsbG1fbW9kZWxzW2NlaWxpbmcoaS81KV0pIHw+DQogICAgICBkcGx5cjo6c2VsZWN0KC1jaGF0X21vZGVsKQ0KICAgIGRmX2NvZWZzW2ksLTFdID0gaXJyQ0FDOjpjb25nZXIua2FwcGEucmF3KG0xKSRlc3QNCiAgICBkZl9jb2Vmc1soaSsxKSwtMV0gPSBpcnJDQUM6OmZsZWlzcy5rYXBwYS5yYXcobTEpJGVzdA0KICAgIGRmX2NvZWZzWyhpKzIpLC0xXSA9IGlyckNBQzo6Z3dldC5hYzEucmF3KG0xKSRlc3QNCiAgICBkZl9jb2Vmc1soaSszKSwtMV0gPSBpcnJDQUM6OmJwLmNvZWZmLnJhdyhtMSkkZXN0DQogICAgZGZfY29lZnNbKGkrNCksLTFdID0gaXJyQ0FDOjprcmlwcGVuLmFscGhhLnJhdyhtMSkkZXN0DQogICAgDQogIH0NCiAgDQogIHJldHVybihkZl9jb2VmcykNCiAgDQp9DQoNCmBgYA0KDQoNCiMjIEN1c3RvbSBGdW5jdGlvbnMgdG8gQ29tcHV0ZSBBZ3JlZW1lbnQgd2l0aCBCZW5jaG1hcmsgeyNjdXN0b21fYWdyZWVtZW50X2Z1bmN0aW9uc30NCg0KSW4gdGhlIGNodW5rIGJlbG93LCB3ZSBwcmVzZW50IHR3byBmdW5jdGlvbnM6ICANCg0KLSBgY2FsY3VsYXRlX2FncmVlbWVudGA6IFRoaXMgZnVuY3Rpb24gY2FsY3VsYXRlcyB0aGUgYWdyZWVtZW50IGJldHdlZW4gdGhlIHJlcGxpY2F0ZXMgYW5kIHRoZSBiZW5jaG1hcmsuIEl0IHRha2VzIHRoZSByZXBsaWNhdGVzIGFuZCB0aGUgYmVuY2htYXJrIGFzIGlucHV0IGFuZCByZXR1cm5zIHRoZSBhZ3JlZW1lbnQgcGVyY2VudGFnZS4gVGhlIGZ1bmN0aW9uIGFsc28gaGFzIGFuIG9wdGlvbmFsIHBhcmFtZXRlciBgbmFgIHRvIGhhbmRsZSBtaXNzaW5nIHZhbHVlcy4gSWYgYG5hID0gMGAsIHRoZSBmdW5jdGlvbiB3aWxsIG5vdCBjb25zaWRlciBtaXNzaW5nIHZhbHVlcyBpbiB0aGUgY2FsY3VsYXRpb24uICANCg0KLSBgZW5zZW1ibGVfcmVwc2A6IFRoaXMgZnVuY3Rpb24gZW5zZW1ibGVzIHRoZSByZXBsaWNhdGVzIGJ5IHNlbGVjdGluZyB0aGUgbW9zdCBjb21tb24gdmFsdWUuIEl0IHRha2VzIHRoZSByZXBsaWNhdGVzIGFzIGlucHV0IGFuZCByZXR1cm5zIHRoZSBtb3N0IGNvbW1vbiB2YWx1ZS4gVGhlIGZ1bmN0aW9uIGFsc28gaGFzIGFuIG9wdGlvbmFsIHBhcmFtZXRlciBgbmFgIHRvIGhhbmRsZSBtaXNzaW5nIHZhbHVlcy4gSWYgYG5hID0gMGAsIHRoZSBmdW5jdGlvbiB3aWxsIG5vdCBjb25zaWRlciBtaXNzaW5nIHZhbHVlcyBpbiB0aGUgY2FsY3VsYXRpb24uIEluIGNhc2Ugb2YgdGllcywgdGhlIGZ1bmN0aW9uIHJhbmRvbWx5IHNlbGVjdHMgb25lIG9mIHRoZSBtb3N0IGNvbW1vbiB2YWx1ZXMuDQoNCmBgYHtyIGNvbXB1dGVfYWdyZWVtZW50X3dpdGhfZ29sZF9zdGFuZGFyZH0NCmNhbGN1bGF0ZV9hZ3JlZW1lbnQgPSBmdW5jdGlvbihyZXBzLCBncm91bmRfdHJ1dGgsIG5hID0gMCkgew0KICBuYV9ybSA9IGRwbHlyOjppZl9lbHNlKG5hID09IDAsIFQsIEYsIG1pc3NpbmcgPSBGKQ0KICANCiAgbWF0Y2hlcyA9IHN1bShyZXBzID09IGdyb3VuZF90cnV0aCwgbmEucm0gPSBuYV9ybSkNCiAgdG90YWxfcmVwcyA9IGxlbmd0aChyZXBzKQ0KICBhZ3JlZW1lbnQgPSBtYXRjaGVzIC8gdG90YWxfcmVwcw0KICANCiAgcmV0dXJuKGFncmVlbWVudCkNCn0NCg0KIyBlbnNlbWJsZSBmdW5jdGlvbiB0byBlbnNlbWJsZSB0aGUgcmVwbGljYXRlcw0KZW5zZW1ibGVfcmVwcyA9IGZ1bmN0aW9uKHJlcHMsIG5hID0gMCl7DQogIA0KICAjIGhhbmRsaW5nIG9mIE5Bcw0KICBuYV9ybSA9IGRwbHlyOjppZl9lbHNlKG5hID09IDAsICJubyIsICJpZmFueSIsIG1pc3NpbmcgPSAiaWZhbnkiKQ0KICANCiAgIyBnZXQgdGhlIG1vc3QgY29tbW9uIHZhbHVlDQogIGZyZXFfdGFibGUgPSB0YWJsZShyZXBzLCB1c2VOQSA9IG5hX3JtKQ0KICBtYXhfZnJlcSA9IG1heChmcmVxX3RhYmxlKQ0KICBtb3N0X2NvbW1vbl92YWx1ZXMgPSBuYW1lcyhmcmVxX3RhYmxlKVtmcmVxX3RhYmxlID09IG1heF9mcmVxXQ0KICANCiAgIyByYW5kb21seSBzZWxlY3Qgb25lIG9mIHRoZSBtb3N0IGNvbW1vbiB2YWx1ZXMNCiAgbW9zdF9jb21tb24gPSBzYW1wbGUobW9zdF9jb21tb25fdmFsdWVzLCAxKQ0KICANCiAgcmV0dXJuKG1vc3RfY29tbW9uKQ0KDQp9DQoNCmBgYA0KDQoNCiMgVGhlIEJpbmFyeSBDbGFzc2lmaWNhdGlvbiBFeHBlcmltZW50IHsjYmluYXJ5X2NsYXNzaWZpY2F0aW9uX2V4cH0NCg0KIyMgRXh0cmFjdGluZyB0aGUgRnVsbCBEYXRhc2V0IHsjZXh0cmFjdF9mdWxsX2RhdGFzZXR9DQoNCldlIHVzZSB0aGUgW3N0b2NrbmV3c2FwaV0oaHR0cHM6Ly9zdG9ja25ld3NhcGkuY29tLykgdG8gZXh0cmFjdCBhcnRpY2xlcyByZWxhdGVkIHRvIHRoZSBzdG9jayBtYXJrZXQuIFdlIGNyYWZ0ZWQgb3VyIHJlcXVlc3QgdG8gZ2V0IGFsbCB0aWNrZXJzIGZyb20gSmFudWFyeSAwNSwgMjAyNSB0byBNYXJjaCAxMywgMjAyNS4gVGhlIGRhdGEgd2FzIHB1bGxlZCBvbiBNYXJjaCAxNCwgMjAyNSBhdCBhcHByb3hpbWF0ZWx5IDEyOjMwIEVEVC4gVGhlIHJlc3VsdGluZyBkYXRhIGZyYW1lIHdhcyBzYXZlZCBhcyBhIFJEUyBmaWxlLiBUbyBydW4gdGhlIGNvZGUgYmVsb3csIHlvdSBuZWVkIHRvIHNldCB0aGUgYHN0b2NrX3Rva2VuYCBlbnZpcm9ubWVudCB2YXJpYWJsZSB0byB5b3VyIEFQSSB0b2tlbi4gSW4gb3VyIGNhc2UsIHdlIHNhdmVkIHRoZSB0b2tlbiBhcyBhbiBlbnZpcm9ubWVudCB2YXJpYWJsZSB1c2luZyB0aGUgYHVzZXRoaXM6OmVkaXRfcl9lbnZpcm9uKHNjb3BlID0gJ3Byb2plY3QnKWAgZnVuY3Rpb24uDQoNCmBgYHtyIGV4dHJhY3Rfc3RvY2tfZGF0YSwgY2FjaGU9VFJVRSwgcmVzdWx0cz0nYXNpcycsIGV2YWw9RkFMU0V9DQojIHB1bGwgdGhlIHN0b2NrX2FwaV90b2tlbiBmcm9tIHRoZSBlbnZpcm9ubWVudCB2YXJpYWJsZQ0Kc3RvY2tfYXBpX3Rva2VuID0gU3lzLmdldGVudigic3RvY2tfdG9rZW4iKQ0KDQojIGNyYWZ0aW5nIHRoZSByZXF1ZXN0IGZvciB0aGUgQVBJDQpyZXF1ZXN0ID0gcGFzdGUwKA0KICAiaHR0cHM6Ly9zdG9ja25ld3NhcGkuY29tL2FwaS92MS9jYXRlZ29yeT8iLCAjIFRoZSBiYXNlIFVSTCBmb3IgdGhlIEFQSQ0KICAic2VjdGlvbj1hbGx0aWNrZXJzIiwgIyBXZSBhcmUgaW50ZXJlc3RlZCBpbiBhbGwgdGlja2Vycw0KICAiJnNlbnRpbWVudD1wb3NpdGl2ZSxuZWdhdGl2ZSIsICMgV2Ugd2FudCBub24gbmV1dHJhbCBzZW50aW1lbnRzDQogICImdHlwZT1hcnRpY2xlIiwgIyBXZSBhcmUgaW50ZXJlc3RlZCBpbiBhcnRpY2xlcw0KICAiJml0ZW1zPTEwMCIsICMgV2Ugd2FudCAxMDAgaXRlbXMgcGVyIHBhZ2UNCiAgIiZkYXRlPTAxMDUyMDI1LTAzMTMyMDI1IiwgIyBUaGUgZGF0ZSByYW5nZQ0KICAiJmV4Y2hhbmdlPU5ZU0UsTkFTREFRIiwgIyBXZSBhcmUgaW50ZXJlc3RlZCBpbiB0aGUgTllTRSBhbmQgTkFTREFRDQogICImY291bnRyeT1VU0EiLCAjIFdlIGFyZSBpbnRlcmVzdGVkIGluIHRoZSBVU0ENCiAgIiZ0b2tlbj0iLCBzdG9ja19hcGlfdG9rZW4sICMgVGhlIEFQSSB0b2tlbiAob3VycyBpcyBzYXZlZCBhcyBhbiBFTlYgVmFyaWFibGUpDQogICImcGFnZT0iKSAjIHRoZSBwYWdlcyB0aGF0IHdlIHdpbGwgaXRlcmF0ZSBvdmVyDQoNCiMgY3JhZnRpbmcgYWxsIHJlcXVlc3RzDQphbGxfcmVxdWVzdHMgPSBwYXN0ZTAocmVxdWVzdCwgMToxMDApDQoNCiMgcHVsbGluZyB0aGUgZGF0YSBmcm9tIHRoZSBBUEkgYW5kIGNsZWFuaW5nIHRoZSBkYXRhIHByaW9yIHRvIHNhdmluZyBpdA0Kc3RvY2tfbmV3c19kZiA9IHB1cnJyOjptYXBfZGYoYWxsX3JlcXVlc3RzLCB+anNvbmxpdGU6OmZyb21KU09OKC54KSRkYXRhKSB8PiANCiAgIyBjb252ZXJ0IHRoZSB0b3BpY3MgYW5kIHRpY2tlcnMgY29scyBpbnRvIGNociAodGhleSBhcmUgbGlzdHMgb2Ygc3RyaW5ncykNCiAgZHBseXI6Om11dGF0ZSh0aWNrZXJzID0gcHVycnI6Om1hcF9jaHIodGlja2VycywgfiBwYXN0ZSgueCwgY29sbGFwc2UgPSAiLCAiKSkpIHw+IA0KICBkcGx5cjo6bXV0YXRlKHRvcGljcyA9IHB1cnJyOjptYXBfY2hyKHRvcGljcywgfiBwYXN0ZSgueCwgY29sbGFwc2UgPSAiLCAiKSkpIHw+IA0KICAjIGNvbnZlcnQgdGhlIGRhdGUgY29sdW1uIGZyb20gY2hhcmFjdGVyIHRvIGRhdGV0aW1lDQogIGRwbHlyOjptdXRhdGUoZGF0ZSA9IGx1YnJpZGF0ZTo6ZG15X2htcyhkYXRlLCB0eiA9ICJBbWVyaWNhL05ld19Zb3JrIikpDQoNCiMgd3JpdGluZyB0aGUgZGF0YSBmcmFtZSBhcyBSRFMgYW5kIENTViBmaWxlcw0KcmVhZHI6OndyaXRlX3JkcyhzdG9ja19uZXdzX2RmLCAiLi4vZGF0YS9zdG9ja19uZXdzX2RhdGEucmRzIikNCnJlYWRyOjp3cml0ZV9jc3Yoc3RvY2tfbmV3c19kZiwgIi4uL2RhdGEvc3RvY2tfbmV3c19kYXRhLmNzdiIpDQpgYGANCg0KYGBge3IgZ2V0X2NvbHVtbl9uYW1lcywgaW5jbHVkZT1GQUxTRX0NCiMgVGhpcyBjb2RlIGNodW5rIGlzIHVzZWQgdG8gZ2V0IHRoZSBjb2x1bW4gbmFtZXMgb2YgdGhlIHN0b2NrX25ld3NfZGYgZGF0YSBmcmFtZQ0KIyBpdCB3aWxsIGJlIHVzZWQgdG8gcHJpbnQgdGhlIGNvbHVtbiBuYW1lcyBuaWNlbHkgaW4gdGhlIHRleHQgYmVsb3cNCiMgd2UgZG8gbm90IHdhbnQgdG8gc2hvdyB0aGUgY29kZSBjaHVuayBpbiB0aGUgZmluYWwgZG9jdW1lbnQgYW5kIGhlbmNlIA0KIyB3ZSBzZXQgaW5jbHVkZT1GQUxTRQ0KDQpzdG9ja19uZXdzX2RmIDwtIHJlYWRyOjpyZWFkX2NzdigiLi4vZGF0YS9zdG9ja19uZXdzX2RhdGEuY3N2IikNCiMgR2V0IGNvbHVtbiBuYW1lcw0KY29sdW1uX25hbWVzID0gbmFtZXMoc3RvY2tfbmV3c19kZikNCg0KIyBDcmVhdGUgYSBzdHJpbmcgd2l0aCBhbGwgY29sdW1uIG5hbWVzIGV4Y2VwdCB0aGUgbGFzdCBvbmUNCmFsbF9idXRfbGFzdCA9IHBhc3RlKGNvbHVtbl9uYW1lc1stbGVuZ3RoKGNvbHVtbl9uYW1lcyldLCBjb2xsYXBzZSA9ICIsICIpDQoNCiMgQWRkIHRoZSBsYXN0IGNvbHVtbiBuYW1lIHdpdGggIiwgYW5kICIgYmVmb3JlIGl0DQpmaW5hbF9zdHJpbmcgPSBzdHJpbmdyOjpzdHJfYyhhbGxfYnV0X2xhc3QsICIsIGFuZCAiLCBjb2x1bW5fbmFtZXNbbGVuZ3RoKGNvbHVtbl9uYW1lcyldKQ0KYGBgDQoNClRoZSBDU1YgZmlsZSBjb250YWluaW5nIHRoZSBzdG9jayBuZXdzIGRhdGEgY2FuIGJlIGFjY2Vzc2VkIFtoZXJlXShodHRwczovL2dpdGh1Yi5jb20vZm1lZ2FoZWQvbGxtX2NvbnNpc3RlbmN5L2Jsb2IvbWFzdGVyL2RhdGEvc3RvY2tfbmV3c19kYXRhLmNzdikuIFRoZSBgc3RvY2tfbmV3c19kZmAgZGF0YSBmcmFtZSBjb250YWlucyBgciBzY2FsZXM6OmNvbW1hKG5yb3coc3RvY2tfbmV3c19kZikpYCByb3dzIGFuZCBgciBuY29sKHN0b2NrX25ld3NfZGYpYCBjb2x1bW5zLiBUaGUgbmFtZXMgb2YgdGhlIGNvbHVtbnMgYXJlOiBgciBmaW5hbF9zdHJpbmdgLiBGdXJ0aGVybW9yZSwgdGhlIHNlbnRpbWVudCBvZiB0aGUgYXJ0aWNsZXMgaXMgc3RvcmVkIGluIHRoZSBgc2VudGltZW50YCBjb2x1bW4uIFRoZSBzZW50aW1lbnQgb2YgdGhvc2UgYXJ0aWNsZXMgaXMgZGl2aWRlZCBpbnRvIHR3byBjYXRlZ29yaWVzOiBgcG9zaXRpdmVgLCBhbmQgYG5lZ2F0aXZlYC4gVGhlaXIgZGlzdHJpYnV0aW9uIGlzIGFzIGZvbGxvd3M6DQoNCmBgYHtyIHNlbnRpbWVudF90YWJsZSwgZWNobz1GQUxTRSwgcmVzdWx0cz0nYXNpcyd9DQpwYW5kZXI6OnBhbmRlcih0YWJsZShzdG9ja19uZXdzX2RmJHNlbnRpbWVudCksIGNvbXBhY3QgPSBUUlVFKQ0KYGBgDQoNCiMjIERvd25zYW1wbGluZyB0aGUgQmluYXJ5IENsYXNzaWZpY2F0aW9uIERhdGFzZXQgeyNkb3duc2FtcGxlX2JpbmFyeV9jbGFzc2lmaWNhdGlvbn0NCg0KSW4gdGhpcyBzdWJzZWN0aW9uLCB3ZSBmaWx0ZXJlZCBvdXQgcm93cyB3aXRoIG11bHRpcGxlIGB0aWNrZXJzYCBpbiB0aGUgdGlja2VycyBjb2x1bW4sIGtlZXBpbmcgb25seSByb3dzIHdpdGggYSBzaW5nbGUgdGlja2VyICh0byBlbnN1cmUgZGF0YSBxdWFsaXR5KS4gQWRkaXRpb25hbGx5LCB3ZSBlbnN1cmVkIHRoZSB1bmlxdWVuZXNzIG9mIHRpY2tlcnMgZHVyaW5nIGRvd24gc2FtcGxpbmcgYnkgc2VsZWN0aW5nIGRpc3RpbmN0IHRpY2tlcnMuIA0KDQpgYGB7ciBkb3duc2FtcGxlX2JpbmFyeV9jbGFzc2lmaWNhdGlvbiwgcmVzdWx0cz0nYXNpcyd9DQpzZXQuc2VlZCgyMDI1KSAjIHNldCB0aGUgc2VlZCBmb3IgcmVwcm9kdWNpYmlsaXR5DQoNCmJpbmFyeV9kYXRhID0gc3RvY2tfbmV3c19kZiB8Pg0KICBkcGx5cjo6ZmlsdGVyKCFzdHJpbmdyOjpzdHJfZGV0ZWN0KHRpY2tlcnMsICIsIikpIHw+ICMga2VlcCByb3dzIHdpdGhvdXQgY29tbWFzIChpLmUuLCBzaW5nbGUgdGlja2VycykNCiAgZHBseXI6Omdyb3VwX2J5KHNlbnRpbWVudCkgfD4gIyBncm91cCBieSBzZW50aW1lbnQNCiAgZHBseXI6OmRpc3RpbmN0KHRpY2tlcnMsIC5rZWVwX2FsbCA9IFRSVUUpIHw+ICAjIGtlZXAgdW5pcXVlIHRpY2tlcnMNCiAgIyByYW5kb21seSBzZWxlY3QgPD02NzUgc2FtcGxlcyBwZXIgc2VudGltZW50IGNsYXNzICh3aWxsIHNlbGVjdCBhbGwgaWYgbiA8IDY3NSkNCiAgZHBseXI6OnNsaWNlX3NhbXBsZShuID0gNjc1KSB8PiANCiAgZHBseXI6OnVuZ3JvdXAoKSAjIHVuZ3JvdXAgdGhlIGRhdGEgZnJhbWUNCg0KIyBzYXZlIHRoZSBkb3duc2FtcGxlZCBkYXRhc2V0IGFzIFJEUyBhbmQgQ1NWIGZpbGVzDQpyZWFkcjo6d3JpdGVfcmRzKGJpbmFyeV9kYXRhLCAiLi4vZGF0YS9iaW5hcnlfY2xhc3NpZmljYXRpb25fZGF0YS5yZHMiKQ0KcmVhZHI6OndyaXRlX2NzdihiaW5hcnlfZGF0YSwgIi4uL2RhdGEvYmluYXJ5X2NsYXNzaWZpY2F0aW9uX2RhdGEuY3N2IikNCmBgYA0KDQpUaGUgcmVzdWx0aW5nIFtiaW5hcnkgY2xhc3NpZmljYXRpb24gZGF0YXNldF0oaHR0cHM6Ly9naXRodWIuY29tL2ZtZWdhaGVkL2xsbV9jb25zaXN0ZW5jeS9ibG9iL21hc3Rlci9kYXRhL2JpbmFyeV9jbGFzc2lmaWNhdGlvbl9kYXRhLmNzdikgDQoNCmNvbnRhaW5zIGByIHNjYWxlczo6Y29tbWEobnJvdyhiaW5hcnlfZGF0YSkpYCBzYW1wbGVzLiBUaGUgZGlzdHJpYnV0aW9uIG9mIHRoZSBzZW50aW1lbnQgY2xhc3NlcyBpcyBhcyBmb2xsb3dzOiANCg0KYGBge3Igc2VudGltZW50X3RhYmxlMiwgZWNobz1GQUxTRSwgcmVzdWx0cz0nYXNpcyd9DQpwYW5kZXI6OnBhbmRlcih0YWJsZShiaW5hcnlfZGF0YSRzZW50aW1lbnQpLCBjb21wYWN0ID0gVFJVRSkNCmBgYA0KDQoNCiMjIExMTS1iYXNlZCBMYWJlbGluZyB7I2xsbV9sYWJlbGluZ19iaW59DQoNCiMjIyBQcm9tcHQgQ29uc3RydWN0aW9uIHsjcHJvbXB0X2NvbnN0cnVjdGlvbl9iaW59DQoNCkZvciBvdXIgbGFiZWxpbmcgdGFzaywgd2UgZGVmaW5lZCBhIHN5c3RlbSBwcm9tcHQgdGhhdCBjb21iaW5lcyAqQ2hhaW4gb2YgVGhvdWdodCogbGVhcm5pbmcgd2l0aCAqZmV3LXNob3QqIGxlYXJuaW5nLiBUaGUgc3lzdGVtIHByb21wdCBwcm92aWRlcyBpbnN0cnVjdGlvbnMgdG8gdGhlIExMTXMgb24gaG93IHRvIGNhdGVnb3JpemUgdGhlIGltcGFjdCBvZiBhIG5ld3MgYXJ0aWNsZSBvbiBhIHN0b2NrJ3MgbmV4dC1kYXkgcmV0dXJuIGFzIGVpdGhlciAiUG9zaXRpdmUiIG9yICJOZWdhdGl2ZSIuIFRoZSBzeXN0ZW0gcHJvbXB0IGFsc28gaW5jbHVkZXMgc2ltdWxhdGVkIGV4YW1wbGVzIChpbnZvbHZpbmcgQml0Y29pbiB3aGljaCBpcyBub3QgaW4gb3VyIHN0b2NrIG5ld3MgZGF0YXNldCkgdG8gZ3VpZGUgdGhlIExMTXMgaW4gdGhlaXIgY2xhc3NpZmljYXRpb24gdGFzay4gVGhlbiwgd2UgZGVmaW5lZCBhIHVzZXIgcHJvbXB0IHRoYXQgcHJlc2VudHMgdGhlIG5ld3MgYXJ0aWNsZSdzIHRpdGxlLCBmdWxsIHRleHQsIGFuZCB0aGUgc3RvY2sgdGlja2VyIHN5bWJvbCB0byB0aGUgTExNcyAodGhlc2Ugd2lsbCBiZSBvYnRhaW5lZCBmcm9tIHRoZSBDU1YgZmlsZSkuIEJvdGggcHJvbXB0cyBhcmUgY29tYmluZWQgaW50byBhIGNoYXQgcHJvbXB0IHRlbXBsYXRlIHRoYXQgd2lsbCBiZSB1c2VkIHRvIGludGVyYWN0IHdpdGggdGhlIExMTXMuDQoNCg0KYGBge3B5dGhvbiBsbG1fYmluYXJ5X3NldHVwLCByZXN1bHRzPSdhc2lzJywgZXZhbD1GQUxTRX0NCiMgZGVmaW5lIHRoZSBzeXN0ZW0gcHJvbXB0IGZvciB0aGUgY2F0ZWdvcml6YXRpb24gdGFzaw0KYmluX3N5c3RlbV9wcm9tcHQgPSAiIiINClRhc2s6IFlvdSB3aWxsIGJlIHByb3ZpZGVkIHdpdGggYSBuZXdzIGFydGljbGUncyB0aXRsZSwgZnVsbCB0ZXh0LCBhbmQgYSBzdG9jayB0aWNrZXIgc3ltYm9sLiBDYXRlZ29yaXplIHRoZSBpbXBhY3Qgb2YgdGhlIG5ld3MgYXJ0aWNsZSBvbiBhIHN0b2NrJ3MgbmV4dC1kYXkgcmV0dXJuIGFzIGVpdGhlciAiUG9zaXRpdmUiIG9yICJOZWdhdGl2ZSIuIA0KDQpJbnN0cnVjdGlvbnM6DQoxLiBSZWFkIHRoZSB0aXRsZSBhbmQgZnVsbCB0ZXh0IG9mIHRoZSBhcnRpY2xlLg0KMi4gQW5hbHl6ZSBob3cgdGhlIGluZm9ybWF0aW9uIG1pZ2h0IGFmZmVjdCB0aGUgY29tcGFueSBhc3NvY2lhdGVkIHdpdGggdGhlIGdpdmVuIHRpY2tlci4NCjMuIElkZW50aWZ5IGtleSBmYWN0b3JzIHN1Y2ggYXMgZmluYW5jaWFsIHBlcmZvcm1hbmNlLCBtYXJrZXQgdHJlbmRzLCBhbm5vdW5jZW1lbnRzLCBhbmQgaW5kdXN0cnkgZGV2ZWxvcG1lbnRzIHRoYXQgbWF5IGluZmx1ZW5jZSBpbnZlc3RvciBzZW50aW1lbnQuDQo0LiBBc3Nlc3MgdGhlIG92ZXJhbGwgdG9uZSBhbmQgaXRzIHBvdGVudGlhbCBpbXBhY3Qgb24gdGhlIHN0b2NrIHByaWNlLg0KDQpDbGFzc2lmaWNhdGlvbiBHdWlkZWxpbmVzOg0KLSAiUG9zaXRpdmUiOiBOZXdzIGxpa2VseSB0byBpbmNyZWFzZSB0aGUgc3RvY2sgcHJpY2UuDQotICJOZWdhdGl2ZSI6IE5ld3MgbGlrZWx5IHRvIGRlY3JlYXNlIHRoZSBzdG9jayBwcmljZS4NCi0gRm9jdXMgb24gdGhlIGltbWVkaWF0ZSBpbXBhY3QgKG5leHQgZGF5IHJldHVybikuDQotIFdlaWdoIHRoZSBpbXBvcnRhbmNlIG9mIHBvc2l0aXZlIHZzLiBuZWdhdGl2ZSBmYWN0b3JzIGlmIHRoZSBhcnRpY2xlIGlzIG1peGVkLg0KDQpPdXRwdXQgRm9ybWF0Og0KLSA8YW5hbHlzaXM+OiBbRGV0YWlsZWQgYW5hbHlzaXMgb2YgdGhlIGFydGljbGXigJlzIGltcGFjdF0NCi0gPGNsYXNzaWZpY2F0aW9uPjogW0ZpbmFsIGNsYXNzaWZpY2F0aW9uOiAiUG9zaXRpdmUiIG9yICJOZWdhdGl2ZSJdDQoNCk5vdGU6IFlvdXIgY2xhc3NpZmljYXRpb24gbXVzdCBzdHJpY3RseSBiZSAiUG9zaXRpdmUiIG9yICJOZWdhdGl2ZSIgYmFzZWQgb24gdGhlIGltbWVkaWF0ZSBleHBlY3RlZCBpbXBhY3QuDQoNCkV4YW1wbGVzOg0KDQpFeGFtcGxlIDE6DQpUaXRsZTogQml0Y29pbiBTdXJnZXMgYXMgTWFqb3IgRmluYW5jaWFsIEluc3RpdHV0aW9uIEFubm91bmNlcyBCVEMgQWRvcHRpb24NClRleHQ6IEluIGEgZ3JvdW5kYnJlYWtpbmcgbW92ZSwgYSBtYWpvciBmaW5hbmNpYWwgaW5zdGl0dXRpb24gYW5ub3VuY2VkIHRoYXQgaXQgd291bGQgc3RhcnQgb2ZmZXJpbmcgQml0Y29pbiBhcyBwYXJ0IG9mIGl0cyBpbnZlc3RtZW50IHBvcnRmb2xpb3MuIFRoaXMgZGVjaXNpb24gaXMgZXhwZWN0ZWQgdG8gc2lnbmlmaWNhbnRseSBpbmNyZWFzZSBpbnN0aXR1dGlvbmFsIGRlbWFuZCBmb3IgQlRDLCBib29zdGluZyBpbnZlc3RvciBjb25maWRlbmNlLg0KVGlja2VyOiBCVEMNCjxhbmFseXNpcz46IFRoZSBhcnRpY2xlIGhpZ2hsaWdodHMgYSBtYWpvciBmaW5hbmNpYWwgaW5zdGl0dXRpb24gYWRvcHRpbmcgQml0Y29pbiwgd2hpY2ggaXMgbGlrZWx5IHRvIGVuaGFuY2UgaW5zdGl0dXRpb25hbCBpbnZlc3RtZW50IGFuZCBkZW1hbmQuIFRoaXMgbmV3cyBwb3NpdGl2ZWx5IGFmZmVjdHMgaW52ZXN0b3Igc2VudGltZW50IGFuZCBzdWdnZXN0cyBhbiBpbW1lZGlhdGUgcG9zaXRpdmUgaW1wYWN0IG9uIEJUQydzIHByaWNlLg0KPGNsYXNzaWZpY2F0aW9uPjogUG9zaXRpdmUNCg0KRXhhbXBsZSAyOg0KVGl0bGU6IEJpdGNvaW4gRmFjZXMgSW5jcmVhc2VkIFJlZ3VsYXRvcnkgU2NydXRpbnkgQW1pZCBGcmF1ZCBDb25jZXJucw0KVGV4dDogUmVwb3J0cyBoYXZlIGVtZXJnZWQgdGhhdCBzZXZlcmFsIGdvdmVybm1lbnRzIGFyZSBwbGFubmluZyB0byBpbXBsZW1lbnQgc3RyaWN0ZXIgcmVndWxhdGlvbnMgb24gY3J5cHRvY3VycmVuY3kgdHJhZGluZywgY2l0aW5nIGNvbmNlcm5zIGFib3V0IGZyYXVkIGFuZCBtYXJrZXQgbWFuaXB1bGF0aW9uLiBUaGUgcmVndWxhdG9yeSBkaXNjdXNzaW9ucyBoYXZlIHNwYXJrZWQgZGViYXRlIGFtb25nIGludmVzdG9ycyByZWdhcmRpbmcgdGhlIGZ1dHVyZSBvZiBCaXRjb2luIGluIGhlYXZpbHkgcmVndWxhdGVkIG1hcmtldHMuDQpUaWNrZXI6IEJUQw0KPGFuYWx5c2lzPjogVGhlIGFydGljbGUgZGlzY3Vzc2VzIHBvdGVudGlhbCByZWd1bGF0b3J5IGFjdGlvbnMgdGhhdCBjb3VsZCBuZWdhdGl2ZWx5IGluZmx1ZW5jZSBtYXJrZXQgc2VudGltZW50IGJ5IHJhaXNpbmcgZmVhcnMgb2YgcmVzdHJpY3RlZCB0cmFkaW5nIGFuZCBoZWlnaHRlbmVkIHNjcnV0aW55LiBUaGlzIG5ld3Mgc3VnZ2VzdHMgYSBsaWtlbHkgbmVnYXRpdmUgaW1wYWN0IG9uIEJUQydzIHByaWNlIGluIHRoZSBpbW1lZGlhdGUgdGVybS4NCjxjbGFzc2lmaWNhdGlvbj46IE5lZ2F0aXZlDQoiIiINCg0KIyBkZWZpbmUgdGhlIHVzZXIgaW5wdXQgc3RyaW5nIHRlbXBsYXRlDQpiaW5fdXNlcl9wcm9tcHQgPSAiIiINCkhlcmUgaXMgdGhlIG5ld3MgYXJ0aWNsZToNCg0KPHRpdGxlPg0Ke3RpdGxlfQ0KPC90aXRsZT4NCg0KPHRleHQ+DQp7dGV4dH0NCjwvdGV4dD4NCg0KVGhlIHN0b2NrIHRpY2tlciBzeW1ib2wgeW91IG5lZWQgdG8gY29uc2lkZXIgaXM6IHt0aWNrZXJzfQ0KIiIiDQoNCiMgY3JlYXRlIHRoZSBjaGF0IHByb21wdCB0ZW1wbGF0ZQ0KYmluX2NoYXRfcHJvbXB0ID0gQ2hhdFByb21wdFRlbXBsYXRlLmZyb21fbWVzc2FnZXMoWw0KICAgICgic3lzdGVtIiwgYmluX3N5c3RlbV9wcm9tcHQpLA0KICAgICgiaHVtYW4iLCBiaW5fdXNlcl9wcm9tcHQpLA0KXSkNCmBgYA0KDQoNCiMjIyBMYWJlbGluZyB0aGUgQmluYXJ5IENsYXNzaWZpY2F0aW9uIERhdGFzZXQgeyNsYWJlbGluZ19iaW59DQoNCkluIHRoaXMgc2VjdGlvbiwgd2Ugd2lsbCB1c2UgdGhlIExMTXMgdG8gbGFiZWwgdGhlIHNlbnRpbWVudCBvZiB0aGUgbmV3cyBhcnRpY2xlcyBpbiBvdXIgYmluYXJ5IGNsYXNzaWZpY2F0aW9uIGRhdGEgc2V0LiBXZSB3aWxsIHVzZSBvdXIgYGdlbmVyYWxpemVkX2NoYXRfY29tcGxldGlvbmAgZnVuY3Rpb24gdG8gaW50ZXJhY3Qgd2l0aCB0aGUgTExNcyBhbmQgZ2VuZXJhdGUgY2hhdCBjb21wbGV0aW9ucyBmb3IgZWFjaCBuZXdzIGFydGljbGUuIFRoZSBjaGF0IGNvbXBsZXRpb25zIHdpbGwgaW5jbHVkZSB0aGUgTExNJ3MgYW5hbHlzaXMgYW5kIGNsYXNzaWZpY2F0aW9uIG9mIHRoZSBuZXdzIGFydGljbGUncyBpbXBhY3Qgb24gdGhlIHN0b2NrJ3MgbmV4dC1kYXkgcmV0dXJuLiBXZSB3aWxsIHJ1biB0aGUgbGFiZWxpbmcgcHJvY2VzcyBmb3IgZWFjaCBMTE0gbW9kZWwgYW5kIHJlcGxpY2F0ZSB0aGUgcHJvY2VzcyB0aHJlZSB0aW1lcyB0byBlbnN1cmUgY29uc2lzdGVuY3kgaW4gdGhlIGxhYmVsaW5nIHJlc3VsdHMuDQoNCmBgYHtweXRob24gYmluYXJ5X2xhYmVsaW5nLCBjYWNoZT1UUlVFLCBldmFsPUZBTFNFfQ0KcmVzID0gZ2VuZXJhbGl6ZWRfY2hhdF9jb21wbGV0aW9uKA0KICBjc3ZfcGF0aCA9ICcuLi9kYXRhL2JpbmFyeV9jbGFzc2lmaWNhdGlvbl9kYXRhLmNzdicsDQogIGNvbHVtbnNfdG9fa2VlcCA9IFsnZGF0ZScsICd0aXRsZScsICd0ZXh0JywgJ3RpY2tlcnMnXSwNCiAgbW9kZWxzID0gbW9kZWxzLA0KICBjaGF0X3Byb21wdF90ZW1wbGF0ZSA9IGJpbl9jaGF0X3Byb21wdCwNCiAgY29sdW1uc19mb3JfY2hhdF9wcm9tcHQgPSBbJ3RpdGxlJywgJ3RleHQnLCAndGlja2VycyddLA0KICBudW1fcmVwbGljYXRlcyA9IDUsDQogIG91dHB1dF9maWxlID0gJy4uL3Jlc3VsdHMvYmluYXJ5X2NsYXNzaWZpY2F0aW9uX3Jlc3VsdHMuY3N2Jw0KICApDQpgYGANCg0KDQpUaGUgbGFiZWxpbmcgcHJvY2VzcyBoYXMgYmVlbiBjb21wbGV0ZWQgZm9yIHRoZSBiaW5hcnkgY2xhc3NpZmljYXRpb24gZGF0YSBzZXQuIFRoZSByZXN1bHRzIGhhdmUgYmVlbiBzYXZlZCB0byBhIENTViBmaWxlLCB3aGljaCBjYW4gYmUgYWNjZXNzZWQgW2hlcmVdKGh0dHBzOi8vZmlnc2hhcmUuY29tL3MvZGE1MjcwZTYyNWVlNjA1ZmZkNDQpLg0KDQojIyMgVGFzayBDb21wbGV0aW9uIFRpbWUgYnkgTW9kZWwNCg0KV2UgcHJlc2VudCBhIGRldGFpbGVkIHN1bW1hcnkgb2YgaW5mZXJlbmNlIGxhdGVuY3kgcGVyZm9ybWFuY2UgYWNyb3NzIG1vZGVscyBiZWxvdy4gDQoNCmBgYHtyIHRpbWUsIGNsYXNzLnNvdXJjZSA9ICdmb2xkLWhpZGUnLCBtZXNzYWdlPUZBTFNFLCBmaWcuYWxpZ249J2NlbnRlcicsIGZpZy53aWR0aD02LCBmaWcuaGVpZ2h0PTgsIGZpZy5mdWxsd2lkdGg9VFJVRX0NCiMgKiBTdW1tYXJ5IFRhYmxlIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KDQpyZXN1bHRzID0gcmVhZHI6OnJlYWRfY3N2KCIuLi9yZXN1bHRzL2JpbmFyeV9jbGFzc2lmaWNhdGlvbl9yZXN1bHRzLmNzdiIpDQoNCnJlc3VsdHMgfD4gIA0KICBkcGx5cjo6ZmlsdGVyKGNoYXRfbW9kZWwgIT0gJ2V4YW9uZS1kZWVwOjIuNGInKSB8PiANCiAgZHBseXI6Om11dGF0ZSgNCiAgICBjaGF0X21vZGVsID0gZm9yY2F0czo6ZmN0X3JlbGV2ZWwoDQogICAgICBjaGF0X21vZGVsLA0KICAgICAgImNsYXVkZS0zLTctc29ubmV0LTIwMjUwMjE5IiwNCiAgICAgICJjbGF1ZGUtMy01LWhhaWt1LTIwMjQxMDIyIiwNCiAgICAgICJncHQtNG8tMjAyNC0xMS0yMCIsDQogICAgICAiZ3B0LTRvLW1pbmktMjAyNC0wNy0xOCIsDQogICAgICAiY29tbWFuZC1yLXBsdXMtMDgtMjAyNCIsDQogICAgICAiY29tbWFuZC1yN2IiLA0KICAgICAgImRlZXBzZWVrLXIxOjdCIiwNCiAgICAgICJkZWVwc2Vlay1yMToxLjVCIiwNCiAgICAgICJnZW1tYTM6MjdCIiwNCiAgICAgICJnZW1tYTM6MUIiLA0KICAgICAgImxsYW1hMy4yOjNCIiwgDQogICAgICAibGxhbWEzLjI6MUIiLA0KICAgICAgJ3BoaTQ6bGF0ZXN0JywNCiAgICAgICdwaGk0LW1pbmknDQogICAgKQ0KICApIHw+IA0KICBkcGx5cjo6Z3JvdXBfYnkoY2hhdF9tb2RlbCkgfD4gDQogIGRwbHlyOjptdXRhdGUoDQogICAgdGltZV9kaWZmID0gY2hhdF9kYXRlIC0gZHBseXI6OmxhZyhjaGF0X2RhdGUpDQogICkgfD4gZHBseXI6OnNlbGVjdChjaGF0X21vZGVsLCBjaGF0X2RhdGUsIHRpbWVfZGlmZikgfD4gDQogIGRwbHlyOjpzdW1tYXJpc2UoDQogICAgbiA9IGRwbHlyOjpuKCksDQogICAgbWluX3QgPSBtaW4odGltZV9kaWZmLCBuYS5ybSA9IFRSVUUpLA0KICAgIHEyNV90ID0gcXVhbnRpbGUodGltZV9kaWZmLCAwLjI1LCBuYS5ybSA9IFRSVUUpLA0KICAgIG1lYW5fdCA9IG1lYW4odGltZV9kaWZmLCBuYS5ybSA9IFRSVUUpLA0KICAgIG1lZF90ID0gbWVkaWFuKHRpbWVfZGlmZiwgbmEucm0gPSBUUlVFKSwNCiAgICBxNzVfdCA9IHF1YW50aWxlKHRpbWVfZGlmZiwgMC43NSwgbmEucm0gPSBUUlVFKSwNCiAgICBwOTVfdCA9IHF1YW50aWxlKHRpbWVfZGlmZiwgMC45NSwgbmEucm0gPSBUUlVFKSwNCiAgICBtYXhfdCA9IG1heCh0aW1lX2RpZmYsIG5hLnJtID0gVFJVRSksDQogICkgfD4gDQogIGRwbHlyOjptdXRhdGUoIA0KICAgIGNvbG9yID0gYyhyZXAoYygnZXhwJywgJ2NoZWFwJyksIDcpICksDQogICAgY29tcGFueSA9IGMoJ0FudGhyb3BpYycsICdBbnRocm9waWMnLCAnQ29oZXJlJywgJ0NvaGVyZScsICdEZWVwU2VlaycsICdEZWVwU2VlaycsDQogICAgICAgICAgICAgICAgJ0dvb2dsZScsICdHb29nbGUnLCAnT3BlbkFJJywgJ09wZW5BSScsICdNZXRhJywgJ01ldGEnLCAnTWljcm9zb2Z0JywgJ01pY3Jvc29mdCcpDQogICAgKSAtPiBzdW1tYXJ5X3RhYmxlDQoNCiMgKiBQbG90dGluZyB0aGUgYm94IHBsb3QgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KDQojIGNyZWF0ZSAiZmFrZSIgc3BhY2VyIGxldmVscyBmb3Igc3BhY2luZyB0aGUgYm94cGxvdDoNCm1vZGVsX2xldmVscyA9IGMoICJzcGFjZXItMCIsDQogICJjbGF1ZGUtMy03LXNvbm5ldCIsICJjbGF1ZGUtMy01LWhhaWt1IiwgICJzcGFjZXItMiIsDQogICJncHQtNG8iLCAgICAgICAgICAgICJncHQtNG8tbWluaSIsICAgICAgICJzcGFjZXItMTBhIiwgInNwYWNlci0xMGIiLA0KICAiY29tbWFuZC1yLXBsdXMiLCAgICAiY29tbWFuZC1yN2IiLCAgICAgICAic3BhY2VyLTQiLA0KICAiZGVlcHNlZWstcjE6N0IiLCAgICAiZGVlcHNlZWstcjE6MS41QiIsICAic3BhY2VyLTYiLA0KICAiZ2VtbWEzOjI3QiIsICAgICAgICAiZ2VtbWEzOjFCIiwgICAgICAgICAic3BhY2VyLTgiLA0KICAibGxhbWEzLjI6M0IiLCAgICAgICAibGxhbWEzLjI6MUIiLCAgICAgICAic3BhY2VyLTEyIiwNCiAgInBoaTQ6bGF0ZXN0IiwgICAgICAgInBoaTQtbWluaSIsICAgICAgICAgInNwYWNlci0xNCINCikNCg0KIyBib3hwbG90IGRhdGE6DQpyZXN1bHRzIHw+ICANCiAgZHBseXI6OmxlZnRfam9pbihzdW1tYXJ5X3RhYmxlLCBieSA9ICJjaGF0X21vZGVsIikgfD4NCiAgZHBseXI6Om11dGF0ZSgNCiAgICB0aW1lX2RpZmYgPSAoY2hhdF9kYXRlIC0gZHBseXI6OmxhZyhjaGF0X2RhdGUpKSB8PiANCiAgICAgIHN0cmluZ3I6OnN0cl9yZW1vdmVfYWxsKCIgc2VjcyIpIHw+IGFzLm51bWVyaWMoKSwNCiAgICBjaGF0X21vZGVsID0gZm9yY2F0czo6ZmN0X3JlY29kZSgNCiAgICAgIGNoYXRfbW9kZWwsDQogICAgICBgY2xhdWRlLTMtNy1zb25uZXRgID0gImNsYXVkZS0zLTctc29ubmV0LTIwMjUwMjE5IiwNCiAgICAgIGBjbGF1ZGUtMy01LWhhaWt1YCA9ICJjbGF1ZGUtMy01LWhhaWt1LTIwMjQxMDIyIiwNCiAgICAgIGBjb21tYW5kLXItcGx1c2AgPSAiY29tbWFuZC1yLXBsdXMtMDgtMjAyNCIsDQogICAgICBgZ3B0LTRvYCA9ICJncHQtNG8tMjAyNC0xMS0yMCIsDQogICAgICBgZ3B0LTRvLW1pbmlgID0gImdwdC00by1taW5pLTIwMjQtMDctMTgiDQogICAgKSB8PiANCiAgICAgIGZhY3RvcihsZXZlbHMgPSBtb2RlbF9sZXZlbHMpDQogICkgfD4gDQogIG5hLm9taXQoKSAtPiBib3hwbG90X2RmDQoNCiMgYW5ub3RhdGlvbiBkZjoNCmxhYmVsX2RmID0gYm94cGxvdF9kZiB8Pg0KICBkcGx5cjo6Z3JvdXBfYnkoY2hhdF9tb2RlbCkgfD4NCiAgZHBseXI6OnN1bW1hcmlzZSgNCiAgICBtaW4gPSByb3VuZChxdWFudGlsZSh0aW1lX2RpZmYsIDApKSwNCiAgICBtZWQgPSByb3VuZChxdWFudGlsZSh0aW1lX2RpZmYsIDAuNSkpLA0KICAgIG1heCA9IHJvdW5kKHF1YW50aWxlKHRpbWVfZGlmZiwgMSkpLA0KICAgIGNvbG9yID0gZHBseXI6OmZpcnN0KGNvbG9yKSwNCiAgICAuZ3JvdXBzID0gImRyb3AiDQogICkgfD4NCiAgdGlkeXI6OnBpdm90X2xvbmdlcigNCiAgICBjb2xzID0gYyhtaW4sIG1lZCwgbWF4KSwNCiAgICBuYW1lc190byA9ICJzdGF0IiwNCiAgICB2YWx1ZXNfdG8gPSAibGFiZWwiDQogICkgfD4NCiAgZHBseXI6OmFycmFuZ2UoZmFjdG9yKGNoYXRfbW9kZWwsIGxldmVscyA9IG1vZGVsX2xldmVscykpIHw+DQogIGRwbHlyOjptdXRhdGUoDQogICAgZ3JvdXBfaW5kZXggPSByZXAoc2VxX2xlbihsZW5ndGgodW5pcXVlKGNoYXRfbW9kZWwpKSksIGVhY2ggPSAzKSwNCiAgICBiYXNlX251ZGdlID0gaWZlbHNlKGdyb3VwX2luZGV4ICUlIDIgPT0gMSwgLTEsIDEpLA0KICAgICMgQXBwbHkgdGlnaHRlciBudWRnZSBmb3IgbWluL21heCwgbG9vc2VyIGZvciBtZWRpYW4NCiAgICB5X251ZGdlID0gZHBseXI6OmNhc2Vfd2hlbigNCiAgICAgIHN0YXQgPT0gIm1lZCIgfiBiYXNlX251ZGdlICogMC43MjUsDQogICAgICBzdGF0ICVpbiUgYygibWluIiwgIm1heCIpIH4gYmFzZV9udWRnZSAqIDAuNQ0KICAgICkNCiAgKSB8PiANCiAgZHBseXI6OmZpbHRlciggIShjaGF0X21vZGVsID09ICJsbGFtYTMuMjoxQiIgJiBzdGF0ID09ICdtaW4nKSApDQoNCg0KIyBkYXRhIGZvciB0aGUgcmVjdGFuZ2xlIGJveGVzOg0KcHJvcHJpZXRhcnlfbW9kZWxzID0gYygiY2xhdWRlLTMtNy1zb25uZXQiLCAiY2xhdWRlLTMtNS1oYWlrdSIsICJncHQtNG8iLCAiZ3B0LTRvLW1pbmkiKQ0KeV9pbmRpY2VzID0gd2hpY2gobW9kZWxfbGV2ZWxzICVpbiUgcHJvcHJpZXRhcnlfbW9kZWxzKQ0KeV9taW4gPSBtaW4oeV9pbmRpY2VzKSAtIDAuNQ0KeV9tYXggPSBtYXgoeV9pbmRpY2VzKSArIDAuNQ0KDQpvcGVuX2FwaSA9IGMoImNvbW1hbmQtci1wbHVzIikNCnlfaW5kaWNlczIgPSB3aGljaChtb2RlbF9sZXZlbHMgJWluJSBvcGVuX2FwaSkNCnlfbWluMiA9IG1pbih5X2luZGljZXMyKSAtIDAuNQ0KeV9tYXgyID0gbWF4KHlfaW5kaWNlczIpICsgMC41DQoNCm9sbGFtYV9tb2RlbHMgPSBjKA0KICAnY29tbWFuZC1yN2InLCAnZGVlcHNlZWstcjE6N0InLCAnZGVlcHNlZWstcjE6MS41QicsICdnZW1tYTM6MjdCJywgJ2dlbW1hMzoxQicsDQogICdsbGFtYTMuMjozQicsICdsbGFtYTMuMjoxQicsICdwaGk0OmxhdGVzdCcsICdwaGk0LW1pbmknDQopDQp5X2luZGljZXMzID0gd2hpY2gobW9kZWxfbGV2ZWxzICVpbiUgb2xsYW1hX21vZGVscykNCnlfbWluMyA9IG1pbih5X2luZGljZXMzKSAtIDAuNQ0KeV9tYXgzID0gbWF4KHlfaW5kaWNlczMpICsgMC41DQoNCg0KIyBjb2xvcnMgZm9yIHlsYWJlbHMNCmNvbG9yX2xvb2t1cCA9IGJveHBsb3RfZGYgfD4NCiAgZHBseXI6OmRpc3RpbmN0KGNoYXRfbW9kZWwsIGNvbG9yKSB8Pg0KICBkcGx5cjo6ZmlsdGVyKCFncmVwbCgiXnNwYWNlciIsIGNoYXRfbW9kZWwpKSB8Pg0KICBkcGx5cjo6bXV0YXRlKA0KICAgIGxhYmVsX2NvbG9yZWQgPSBkcGx5cjo6Y2FzZV93aGVuKA0KICAgICAgY29sb3IgPT0gImV4cCIgfiBwYXN0ZTAoIjxzcGFuIHN0eWxlPSdjb2xvcjojQzMxNDJEJz4iLCBjaGF0X21vZGVsLCAiPC9zcGFuPiIpLA0KICAgICAgY29sb3IgPT0gImNoZWFwIiB+IHBhc3RlMCgiPHNwYW4gc3R5bGU9J2NvbG9yOiM4MDgwODAnPiIsIGNoYXRfbW9kZWwsICI8L3NwYW4+IiksDQogICAgICBUUlVFIH4gY2hhdF9tb2RlbA0KICAgICkNCiAgKQ0KDQptb2RlbF9sYWJlbHMgPSBzZXROYW1lcyhjb2xvcl9sb29rdXAkbGFiZWxfY29sb3JlZCwgY29sb3JfbG9va3VwJGNoYXRfbW9kZWwpDQoNCiMgQWRkIGVtcHR5IGxhYmVscyBmb3Igc3BhY2Vycw0Kc3BhY2VyX2xhYmVscyA9IHJlcCgiIiwgc3VtKGdyZXBsKCJec3BhY2VyIiwgbW9kZWxfbGV2ZWxzKSkpDQpuYW1lcyhzcGFjZXJfbGFiZWxzKSA9IG1vZGVsX2xldmVsc1tncmVwbCgiXnNwYWNlciIsIG1vZGVsX2xldmVscyldDQoNCiMgTWVyZ2UNCmZ1bGxfbGFiZWxzID0gYyhtb2RlbF9sYWJlbHMsIHNwYWNlcl9sYWJlbHMpDQpmdWxsX2xhYmVscyA9IGZ1bGxfbGFiZWxzW21vZGVsX2xldmVsc10gICMgUmVvcmRlciB0byBtYXRjaCBheGlzDQoNCg0KIyBwbG90Og0KYm94cGxvdF9kZiB8Pg0KICBkcGx5cjo6bXV0YXRlKGNoYXRfbW9kZWwgPSBmYWN0b3IoY2hhdF9tb2RlbCwgbGV2ZWxzID0gbW9kZWxfbGV2ZWxzKSkgfD4gDQogIGdncGxvdDI6OmdncGxvdChnZ3Bsb3QyOjphZXMoeSA9IGNoYXRfbW9kZWwsIHggPSB0aW1lX2RpZmYpKSArDQogIGdncGxvdDI6Omdlb21fYm94cGxvdChnZ3Bsb3QyOjphZXMoY29sb3IgPSBjb2xvciksIHNpemUgPTAuNSkgKyAgDQogIA0KICAjIGFkZGluZyBibGFua3MgYmV0d2VlbiBwYWlycyBvZiBtb2RlbHMNCiAgZ2dwbG90Mjo6Z2VvbV9ibGFuayhkYXRhID0gZGF0YS5mcmFtZShjaGF0X21vZGVsID0gbW9kZWxfbGV2ZWxzLCB0aW1lX2RpZmYgPSBOQSkpICsgDQogIA0KICAjIHVzZSBnZ3Bsb3QyOjpzdGF0X3N1bW1hcnkgdG8gYW5ub3RhdGUgdGhlIG1heCBmb3IgZWFjaCBtb2RlbA0KICBnZ3Bsb3QyOjpnZW9tX3RleHQoDQogICAgZGF0YSA9IGxhYmVsX2RmLA0KICAgIGdncGxvdDI6OmFlcyh4ID0gbGFiZWwsIHkgPSBjaGF0X21vZGVsLCBsYWJlbCA9IGxhYmVsLCBjb2xvciA9IGNvbG9yKSwNCiAgICBwb3NpdGlvbiA9IGdncGxvdDI6OnBvc2l0aW9uX251ZGdlKHkgPSBsYWJlbF9kZiR5X251ZGdlKSwNCiAgICBzaXplID0gMi43NSwNCiAgKSArDQogIA0KICBnZ3Bsb3QyOjpzY2FsZV94X2xvZzEwKCkgKw0KICBnZ3Bsb3QyOjpzY2FsZV95X2Rpc2NyZXRlKA0KICAgIGxpbWl0cyA9IG1vZGVsX2xldmVscywNCiAgICBsYWJlbHMgPSBmdWxsX2xhYmVscw0KICApICsNCiAgZ2dwbG90Mjo6c2NhbGVfY29sb3JfbWFudWFsKHZhbHVlcyA9IGMoImV4cCIgPSAiI0MzMTQyRCIsICJjaGVhcCIgPSAiIzgwODA4MCIpKSArDQogIA0KICBnZ3Bsb3QyOjphbm5vdGF0ZSgicmVjdCIsDQogICAgICAgICAgICAgeG1pbiA9IDAuOSwgeG1heCA9IDEyMDAsDQogICAgICAgICAgICAgeW1pbiA9IHlfbWluLTAuOCwgeW1heCA9IHlfbWF4KzAuNiwNCiAgICAgICAgICAgICBhbHBoYSA9IDAuMDIsDQogICAgICAgICAgICAgZmlsbCA9IE5BLA0KICAgICAgICAgICAgIGNvbG9yID0gImJsYWNrIiwNCiAgICAgICAgICAgICBsaW5ldHlwZSA9IDMNCiAgKSArDQogIGdncGxvdDI6OmFubm90YXRlKCJ0ZXh0IiwNCiAgICAgICAgICAgeCA9IDMxMCwNCiAgICAgICAgICAgeSA9IHlfbWF4ICsgLjM1LCAgIyBqdXN0IGluc2lkZSB0aGUgYm94DQogICAgICAgICAgIGxhYmVsID0gIlByb3ByaWV0YXJ5IExMTXNcbiAodmlhIEFQSSkiLA0KICAgICAgICAgICBoanVzdCA9IDAuNSwNCiAgICAgICAgICAgdmp1c3QgPSAxLA0KICAgICAgICAgICBmb250ZmFjZSA9ICJib2xkIiwNCiAgICAgICAgICAgc2l6ZSA9IDMNCiAgKSArDQogIGdncGxvdDI6OmFubm90YXRlKCJyZWN0IiwNCiAgICAgICAgICAgICAgICAgICAgeG1pbiA9IDAuOSwgeG1heCA9IDEyMDAsDQogICAgICAgICAgICAgICAgICAgIHltaW4gPSB5X21pbjMtMS42LCB5bWF4ID0geV9tYXgzKzAuNiwNCiAgICAgICAgICAgICAgICAgICAgYWxwaGEgPSAwLjAyLA0KICAgICAgICAgICAgICAgICAgICBjb2xvciA9ICJibGFjayIsDQogICAgICAgICAgICAgICAgICAgIGZpbGwgPSBOQSwNCiAgICAgICAgICAgICAgICAgICAgbGluZXR5cGUgPSAyDQogICkgKw0KICBnZ3Bsb3QyOjphbm5vdGF0ZSgidGV4dCIsDQogICAgICAgICAgICAgICAgICAgIHggPSAxMjAsDQogICAgICAgICAgICAgICAgICAgIHkgPSB5X21heDMrMC4zNSAsICAjIGp1c3QgaW5zaWRlIHRoZSBib3gNCiAgICAgICAgICAgICAgICAgICAgbGFiZWwgPSAiT3BlbiBXZWlnaHQgTExNcyB2aWEgT2xsYW1hXG4gKGNvbW1hbmQtci1wbHVzIHZpYSBBUEkpIiwNCiAgICAgICAgICAgICAgICAgICAgaGp1c3QgPSAwLjUsDQogICAgICAgICAgICAgICAgICAgIHZqdXN0ID0gMSwNCiAgICAgICAgICAgICAgICAgICAgZm9udGZhY2UgPSAiYm9sZCIsDQogICAgICAgICAgICAgICAgICAgIHNpemUgPSAzDQogICkgKw0KICBnZ3Bsb3QyOjpsYWJzKA0KICAgIHRpdGxlID0gIlRhc2sgQ29tcGxldGlvbiBUaW1lIGJ5IE1vZGVsIiwNCiAgICBzdWJ0aXRsZSA9ICI8c3BhbiBzdHlsZT0nY29sb3I6IzgwODA4MCc+Q2hlYXBlcjwvc3Bhbj4gdnMuIDxzcGFuIHN0eWxlPSdjb2xvcjojQzMxNDJEJz5tb3JlIGV4cGVuc2l2ZSAodGltZSwgY29zdCkgPC9zcGFuPiBMTE1zIGJ5IGNvbXBhbnkiLA0KICAgIHkgPSAiIiwNCiAgICB4ID0gIlRhc2sgQ29tcGxldGlvbiBUaW1lIGluIFNlY29uZHMgKExvZyBTY2FsZSkiLA0KICAgIGNhcHRpb24gPSAnQW5ub3RhdGVkIHdpdGggbWluLCBtZWRpYW4sIGFuZCBtYXggdGltZXMgYmFzZWQgb24gbj02LDc1MCBhbm5vdGF0aW9ucyBwZXIgbW9kZWwuJw0KICAgICAgICkgKw0KICBnZ3Bsb3QyOjp0aGVtZV9jbGFzc2ljKCkgKw0KICBnZ3Bsb3QyOjp0aGVtZSgNCiAgICBwbG90LnRpdGxlID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGhqdXN0ID0gMC41LCBmYWNlID0gImJvbGQiLCBzaXplID0gMTEpLA0KICAgIHBsb3Quc3VidGl0bGUgPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oaGp1c3QgPSAxLCBmYWNlID0gImJvbGQiLCBzaXplID0gMTApLA0KICAgIGF4aXMudGl0bGUueCA9IGdndGV4dDo6ZWxlbWVudF9tYXJrZG93bihoanVzdCA9IDAuNSwgZmFjZSA9ICJib2xkIiwgc2l6ZSA9IDkpLA0KICAgIGF4aXMudGl0bGUueSA9IGdndGV4dDo6ZWxlbWVudF9tYXJrZG93bihoanVzdCA9IDAuNSwgZmFjZSA9ICJib2xkIiwgc2l6ZSA9IDkpLA0KICAgIGF4aXMudGV4dC54ID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGhqdXN0ID0gMC41LCBzaXplID0gOSwgZmFjZSA9ICdib2xkJyksDQogICAgYXhpcy50ZXh0LnkgPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oaGp1c3QgPSAxLCBzaXplID0gOSwgZmFjZSA9ICdib2xkJyksDQogICAgcGFuZWwuZ3JpZC5tYWpvciA9IGdncGxvdDI6OmVsZW1lbnRfYmxhbmsoKSwNCiAgICBwYW5lbC5ncmlkLm1pbm9yID0gZ2dwbG90Mjo6ZWxlbWVudF9ibGFuaygpLA0KICAgIGxlZ2VuZC5wb3NpdGlvbiA9ICJub25lIiwNCiAgICBwbG90LmNhcHRpb24gPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oaGp1c3QgPSAwLjk1LCBzaXplID0gOCksDQogICAgYXhpcy50aWNrcy55ID0gZ2dwbG90Mjo6ZWxlbWVudF9ibGFuaygpDQogICkNCmBgYA0KDQoNCiMjIyBFeHRyYWN0aW5nIHRoZSBMTE0gTGFiZWxzIGFuZCBDcmVhdGluZyBhIERhdGEgRnJhbWUgZm9yIEFuYWx5c2lzIHsjZXh0cmFjdF9sbG1fbGFiZWxzfQ0KDQpJbiB0aGlzIHNlY3Rpb24sIHdlIHdpbGwgZXh0cmFjdCB0aGUgTExNIGxhYmVscyBmcm9tIHRoZSBiaW5hcnkgY2xhc3NpZmljYXRpb24gcmVzdWx0cyBhbmQgY3JlYXRlIGEgZGF0YSBmcmFtZSBmb3IgZnVydGhlciBhbmFseXNpcy4gV2Ugd2lsbCBleHRyYWN0IHRoZSBMTE0gbGFiZWxzIGZyb20gdGhlIGNoYXQgcmVzcG9uc2VzIGFuZCBjcmVhdGUgYSBuZXcgY29sdW1uIGluIHRoZSBkYXRhIGZyYW1lIGZvciBlYWNoIExMTSBtb2RlbC4gV2Ugd2lsbCBhbHNvIGNhbGN1bGF0ZSB0aGUgZnJlcXVlbmN5IG9mIHRoZSBtb3N0IGNvbW1vbiBsYWJlbCBhY3Jvc3MgcmVwbGljYXRlcyB0byBhc3Nlc3MgdGhlIGNvbnNpc3RlbmN5IG9mIHRoZSBMTE1zLg0KDQpgYGB7ciBiaW5hcnlfY2xhc3NpZmljYXRpb25fbGFiZWxzLCByZXN1bHRzPSdob2xkJywgZXZhbD1GQUxTRX0NCnBhY21hbjo6cF9sb2FkKGlyckNBQywgc3RyaW5nciwgdGlkeXZlcnNlKQ0KDQojIGhlbHBlciBmdW5jdGlvbiB0byBjYWxjdWxhdGUgdGhlIGZyZXF1ZW5jeSBvZiB0aGUgbW9zdCBjb21tb24gdmFsdWUNCm1vZGVfZnJlcXVlbmN5ID0gZnVuY3Rpb24oeCwgZHJvcF9uYT1GKSB7DQogICMgcmV0dXJuIDAgaWYgYWxsIHZhbHVlcyBhcmUgTkENCiAgaWYgKGFsbChpcy5uYSh4KSkpIHsNCiAgICByZXR1cm4oMCkNCiAgfQ0KICAjIGNhbGN1bGF0ZSB0aGUgZnJlcXVlbmN5IHRhYmxlDQogIGZyZXFfdGFibGUgPSB0YWJsZSh4KQ0KICBpZiAoZHJvcF9uYT09Ril7DQogICAgIyByZXR1cm4gdGhlIGZyZXF1ZW5jeSBvZiB0aGUgbW9zdCBjb21tb24gdmFsdWUNCiAgICByZXR1cm4obWF4KGZyZXFfdGFibGUpIC8gbGVuZ3RoKHgpKQ0KICB9ZWxzZXsNCiAgICByZXR1cm4obWF4KGZyZXFfdGFibGUpL3N1bShmcmVxX3RhYmxlKSkNCiAgfQ0KfQ0KDQojIHJlYWQgdGhlIGJpbmFyeSBjbGFzc2lmaWNhdGlvbiByZXN1bHRzIGZyb20gdGhlIENTViBmaWxlDQpiaW5hcnlfY2xhc3NpZmljYXRpb25fcmVzdWx0cyA9IHJlYWRyOjpyZWFkX2NzdigiLi4vcmVzdWx0cy9iaW5hcnlfY2xhc3NpZmljYXRpb25fcmVzdWx0cy5jc3YiKQ0KYmluYXJ5X2NsYXNzaWZpY2F0aW9uX3Jlc3VsdHMgPSBiaW5hcnlfY2xhc3NpZmljYXRpb25fcmVzdWx0cyB8Pg0KICBmaWx0ZXIoIShjaGF0X21vZGVsICVpbiUgYygiZXhhb25lLWRlZXA6Mi40YiIpKSkNCg0KIyBhbGxvd2FibGUgd29yZHMgZm9yIHRoZSBjbGFzc2lmaWNhdGlvbg0KdmFsaWRfd29yZHMgPC0gYygiTmVnYXRpdmUiLCAiUG9zaXRpdmUiLCAiTmV1dHJhbCIsICJNaXhlZCIsICJJbmNvbmNsdXNpdmUiLCAiSW5zdWZmaWNpZW50IiwgIkltcG9zc2libGUiLCAiQW1iaWd1b3VzIiwgIkluZGV0ZXJtaW5hdGUiKQ0KDQojIGV4dHJhY3QgdGhlIExMTSBsYWJlbHMgZnJvbSB0aGUgY2hhdCByZXNwb25zZXMNCmJpbmFyeV9jbGFzc2lmaWNhdGlvbl9yZXN1bHRzID0gYmluYXJ5X2NsYXNzaWZpY2F0aW9uX3Jlc3VsdHMgfD4NCiAgbXV0YXRlKGxsbV9sYWJlbCA9IHNhcHBseShjaGF0X3Jlc3BvbnNlLCBleHRyYWN0X2NsYXNzaWZpY2F0aW9uLCB2YWxpZF93b3JkcyA9IHZhbGlkX3dvcmRzKSwNCiAgICAgICAgIGxsbV9sYWJlbCA9IHVubmFtZShsbG1fbGFiZWwpLA0KICAgICAgICAgbGxtX2xhYmVsID0gc3RyX3RvX3RpdGxlKGxsbV9sYWJlbCksDQogICAgICAgICBsbG1fbGFiZWwgPSBpZmVsc2UobGxtX2xhYmVsICVpbiUgYygiTmV1dHJhbCIsICJNaXhlZCIsICJJbmNvbmNsdXNpdmUiLCAiSW5zdWZmaWNpZW50IiwgIkltcG9zc2libGUiLCAiQW1iaWd1b3VzIiwgIkluZGV0ZXJtaW5hdGUiKSwgTkEsIGxsbV9sYWJlbCksDQogICAgICAgICBsbG1fbGFiZWwgPSBhcy5mYWN0b3IobGxtX2xhYmVsKQ0KICAgICAgICAgKSB8Pg0KICAjIHNlbGVjdCByZWxldmFudCBjb2x1bW5zDQogIGRwbHlyOjpzZWxlY3QoZGF0ZSwgdGl0bGUsIHRleHQsIHRpY2tlcnMsIGNoYXRfbW9kZWwsIGNoYXRfcmVwbGljYXRlLCBsbG1fbGFiZWwpDQoNCiMgcHJpbnQgdGhlIGZyZXF1ZW5jeSBvZiBsbG1fbGFiZWwNCnByaW50KHRhYmxlKGJpbmFyeV9jbGFzc2lmaWNhdGlvbl9yZXN1bHRzJGxsbV9sYWJlbCwgdXNlTkEgPSAiYWx3YXlzIikpDQoNCiMgcGl2b3QgdGhlIGRhdGEgdG8gd2lkZSBmb3JtYXQgZm9yIGVhc2llciBjb21wYXJpc29uIChhbmQgYXBwZW5kIHRoZSByZXBsaWNhdGUgbnVtYmVyIGluIG5hbWUpDQpiaW5hcnlfYW5hbHlzaXNfZGYgPC0gYmluYXJ5X2NsYXNzaWZpY2F0aW9uX3Jlc3VsdHMgfD4gDQogICMgcGl2b3QgdGhlIGRhdGEgdG8gd2lkZSBmb3JtYXQgZm9yIGVhc2llciBjb21wYXJpc29uIChhbmQgYXBwZW5kIHRoZSByZXBsaWNhdGUgbnVtYmVyIGluIG5hbWUpDQogIHRpZHlyOjpwaXZvdF93aWRlcihuYW1lc19mcm9tID0gY2hhdF9yZXBsaWNhdGUsIHZhbHVlc19mcm9tID0gbGxtX2xhYmVsLCBuYW1lc19wcmVmaXggPSAicmVwXyIpIHw+DQogICMgY29tcHV0ZSB0aGUgZnJlcXVlbmN5IG9mIHRoZSBtb3N0IGZyZXF1ZW50IHZhbHVlIGFjcm9zcyByZXBsaWNhdGVzDQogIGRwbHlyOjpyb3d3aXNlKCkgfD4NCiAgZHBseXI6Om11dGF0ZSgNCiAgICBwZXJjZW50X2FncmVlbWVudCA9IG1vZGVfZnJlcXVlbmN5KA0KICAgICAgZHBseXI6OmNfYWNyb3NzKHRpZHlzZWxlY3Q6OnN0YXJ0c193aXRoKCJyZXBfIikpDQogICAgKSAqIDEwMCwNCiAgICBwZXJjZW50X2FncmVlbWVudF9kcm9wX25hID0gbW9kZV9mcmVxdWVuY3koDQogICAgICBkcGx5cjo6Y19hY3Jvc3ModGlkeXNlbGVjdDo6c3RhcnRzX3dpdGgoInJlcF8iKSksIGRyb3BfbmE9VA0KICAgICkgKiAxMDANCiAgKSB8Pg0KICBkcGx5cjo6dW5ncm91cCgpICMgdW5ncm91cCBhZnRlciByb3d3aXNlIG9wZXJhdGlvbg0KDQojIHNhdmUgdGhlIGFuYWx5c2lzIGRhdGFmcmFtZSBhcyBSRFMgYW5kIENTViBmaWxlcw0KcmVhZHI6OndyaXRlX3JkcyhiaW5hcnlfYW5hbHlzaXNfZGYsICIuLi9yZXN1bHRzL2JpbmFyeV9hbmFseXNpc19kZi5yZHMiKQ0KcmVhZHI6OndyaXRlX2NzdihiaW5hcnlfYW5hbHlzaXNfZGYsICIuLi9yZXN1bHRzL2JpbmFyeV9hbmFseXNpc19kZi5jc3YiKQ0KDQpgYGANCg0KVGhlIGFuYWx5c2lzIGRhdGFmcmFtZSBmb3IgdGhlIGJpbmFyeSBjbGFzc2lmaWNhdGlvbiBleHBlcmltZW50IGhhcyBiZWVuIGNyZWF0ZWQuIFRoZSBkYXRhZnJhbWUgY29udGFpbnMgdGhlIExMTSBsYWJlbHMgZm9yIGVhY2ggbW9kZWwgYW5kIHJlcGxpY2F0ZSwgYWxvbmcgd2l0aCB0aGUgZnJlcXVlbmN5IG9mIHRoZSBtb3N0IGNvbW1vbiBsYWJlbCBhY3Jvc3MgcmVwbGljYXRlcy4gVGhlIHJlc3VsdHMgY2FuIGJlIGFjY2Vzc2VkIGluIHRoZSBbYmluYXJ5X2FuYWx5c2lzX2RmIENTViBmaWxlXShodHRwczovL2dpdGh1Yi5jb20vZm1lZ2FoZWQvbGxtX2NvbnNpc3RlbmN5L2Jsb2IvbWFpbi9yZXN1bHRzL2JpbmFyeV9hbmFseXNpc19kZi5jc3YpLg0KDQoNCiMjIyBEaXN0cmlidXRpb24gb2YgTExNcyBmb3IgQmluYXJ5IENsYXNzaWZpY2F0aW9uDQoNCkluIHRoaXMgc2VjdGlvbiwgd2Ugc3R1ZHkgdGhlIGRpc3RyaWJ1dGlvbiBvZiBMTE0gbGFiZWxzIGZvciBiaW5hcnkgY2xhc3NpZmljYXRpb24uIFRoZSBmaWd1cmUgYmVsb3csIHRoZSAiaW52YWxpZCIgbGFiZWwgcmVwcmVzZW50cyB0aGUgaW5jb25zaXN0ZW5jeSBiZXR3ZWVuIG91ciBwcm9tcHQgKHdoaWNoIGV4cGxpY2l0bHkgaW5zdHJ1Y3RlZCB0aGUgTExNIHRvIHJlc3RyaWN0IGl0cyBjbGFzc2lmaWNhdGlvbiB0byBlaXRoZXIgInBvc2l0aXZlIiBvciAibmVnYXRpdmUiKSBhbmQgdGhlIGdlbmVyYXRlZCBjbGFzc2lmaWNhdGlvbiwgd2hpY2ggc29tZXRpbWVzIGluY2x1ZGVkIHRlcm1zIHN1Y2ggYXMgIm5ldXRyYWwiIG9yICJ1bnN1cmUiLg0KDQoNCmBgYHtyIExMTV9sYWJlbHNfZGlzdHJpYnV0aW9uLCBjbGFzcy5zb3VyY2UgPSAnZm9sZC1oaWRlJywgbWVzc2FnZT1GQUxTRSwgZmlnLmFsaWduPSdjZW50ZXInLCBmaWcud2lkdGg9NiwgZmlnLmhlaWdodD04LCBmaWcuZnVsbHdpZHRoPVRSVUV9DQoNCiMgbGlicmFyeSh0aWR5dmVyc2UpDQoNCiMgKiBHZXR0aW5nIHRoZSBEYXRhIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KbWlhbWlyZWQgPSAnI0MzMTQyRCcNCmJpbmFyeV9jbGFzc2lmaWNhdGlvbl9yZXN1bHRzID0gDQogIHJlYWRyOjpyZWFkX2NzdigiLi4vcmVzdWx0cy9iaW5hcnlfYW5hbHlzaXNfZGYuY3N2IikgfD4gDQogIGRwbHlyOjpzZWxlY3QoDQogICAgZGF0ZSwgdGl0bGUsIHRpY2tlcnM6cmVwXzUNCiAgKSB8PiANCiAgdGlkeXI6OnBpdm90X2xvbmdlcigNCiAgICBjb2xzID0gcmVwXzE6cmVwXzUsDQogICAgbmFtZXNfdG8gPSAicmVwIiwNCiAgICB2YWx1ZXNfdG8gPSAibGxtX2xhYmVsIg0KICApIHw+ICANCiAgZHBseXI6OmZpbHRlcihjaGF0X21vZGVsICE9ICdleGFvbmUtZGVlcDoyLjRiJykgDQoNCg0KIyAqIFN0dWZmIE5lZWRlZCBmb3IgQ29sb3JpbmcgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQoNCiMgKiBDaGF0IE1vZGVscyAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KDQojIEFzc2lnbiBhbHRlcm5hdGluZyBjb2xvcnMgdG8gY2hhdF9tb2RlbCBsYWJlbHMNCmNoYXRfbW9kZWxzID0gYygNCiAgInBoaTQtbWluaSIsDQogICJwaGk0OmxhdGVzdCIsDQogICJsbGFtYTMuMjoxQiIsDQogICJsbGFtYTMuMjozQiIsIA0KICAiZ2VtbWEzOjFCIiwNCiAgImdlbW1hMzoyN0IiLA0KICAiZGVlcHNlZWstcjE6MS41QiIsDQogICJkZWVwc2Vlay1yMTo3QiIsDQogICJjb21tYW5kLXI3YiIsDQogICJjb21tYW5kLXItcGx1cy0wOC0yMDI0IiwNCiAgImdwdC00by1taW5pLTIwMjQtMDctMTgiLA0KICAiZ3B0LTRvLTIwMjQtMTEtMjAiLA0KICAiY2xhdWRlLTMtNS1oYWlrdS0yMDI0MTAyMiIsDQogICJjbGF1ZGUtMy03LXNvbm5ldC0yMDI1MDIxOSINCikNCg0KbW9kZWxfYWJicmV2ID0gYygNCiAgInBoaTQtbWluaSIgICAgICAgICAgICAgICAgID0gInBoaTQtbWluaSIsDQogICJwaGk0OmxhdGVzdCIgICAgICAgICAgICAgICA9ICJwaGk0OmxhdGVzdCIsDQogICJsbGFtYTMuMjoxQiIgICAgICAgICAgICAgICA9ICJsbGFtYTMuMjoxQiIsDQogICJsbGFtYTMuMjozQiIgICAgICAgICAgICAgICA9ICJsbGFtYTMuMjozQiIsDQogICJnZW1tYTM6MUIiICAgICAgICAgICAgICAgICA9ICJnZW1tYTM6MUIiLA0KICAiZ2VtbWEzOjI3QiIgICAgICAgICAgICAgICAgPSAiZ2VtbWEzOjI3QiIsDQogICJkZWVwc2Vlay1yMToxLjVCIiAgICAgICAgICA9ICJkZWVwc2Vlay1yMToxLjVCIiwNCiAgImRlZXBzZWVrLXIxOjdCIiAgICAgICAgICAgID0gImRlZXBzZWVrLXIxOjdCIiwNCiAgImNvbW1hbmQtcjdiIiAgICAgICAgICAgICAgID0gImNvbW1hbmQtcjdiIiwNCiAgImNvbW1hbmQtci1wbHVzLTA4LTIwMjQiICAgID0gImNvbW1hbmQtci1wbHVzIiwNCiAgImdwdC00by1taW5pLTIwMjQtMDctMTgiICAgID0gImdwdC00by1taW5pIiwNCiAgImdwdC00by0yMDI0LTExLTIwIiAgICAgICAgID0gImdwdC00byIsDQogICJjbGF1ZGUtMy01LWhhaWt1LTIwMjQxMDIyIiA9ICJjbGF1ZGUtMy01LWhhaWt1IiwNCiAgImNsYXVkZS0zLTctc29ubmV0LTIwMjUwMjE5Ij0gImNsYXVkZS0zLTctc29ubmV0Ig0KKQ0KDQojIEFwcGx5IGFsdGVybmF0aW5nIGNvbG9yIHNwYW4gdGFncw0KbGFiZWxfY29sb3JzID0gYygiTmVnYXRpdmUiID0gIiMxRjc4QjQiLCAiSW52YWxpZCIgPSAiI0IwQjBCMCIsICJQb3NpdGl2ZSIgPSAiI0E2Q0VFMyIpDQoNCg0KIyAqIFBsb3R0aW5nIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KcGxvdHMgPSBwdXJycjo6aW1hcChjaGF0X21vZGVscywgZnVuY3Rpb24obW9kZWwsIGlkeCkgew0KICBjb2xvciA9IGlmIChpZHggJSUgMiA9PSAxKSAiIzgwODA4MCIgIGVsc2UgbWlhbWlyZWQNCiAgbW9kZWxfc2hvcnQgPSBtb2RlbF9hYmJyZXZbW21vZGVsXV0NCiAgDQogIGJpbmFyeV9jbGFzc2lmaWNhdGlvbl9yZXN1bHRzIHw+DQogICAgZHBseXI6OmZpbHRlcihjaGF0X21vZGVsID09IG1vZGVsKSB8Pg0KICAgIGRwbHlyOjptdXRhdGUoDQogICAgICBsbG1fbGFiZWwgPSBmb3JjYXRzOjpmY3RfZXhwYW5kKGxsbV9sYWJlbCwgIkludmFsaWQiKSwNCiAgICAgIGxsbV9sYWJlbCA9IGZvcmNhdHM6OmZjdF9uYV92YWx1ZV90b19sZXZlbChsbG1fbGFiZWwsICJJbnZhbGlkIiksDQogICAgICBsbG1fbGFiZWwgPSBmb3JjYXRzOjpmY3RfcmVsZXZlbChsbG1fbGFiZWwsICJQb3NpdGl2ZSIsICJJbnZhbGlkIiwgIk5lZ2F0aXZlIikNCiAgICApIHw+DQogICAgZHBseXI6Omdyb3VwX2J5KGxsbV9sYWJlbCkgfD4NCiAgICBkcGx5cjo6c3VtbWFyaXplKGNvdW50ID0gZHBseXI6Om4oKSkgfD4NCiAgICBkcGx5cjo6bXV0YXRlKHBlcmNlbnQgPSBjb3VudCAvIHN1bShjb3VudCkgKiAxMDApIHw+DQogICAgZ2dwbG90Mjo6Z2dwbG90KCBnZ3Bsb3QyOjphZXMoeCA9IGxsbV9sYWJlbCwgeSA9IHBlcmNlbnQsIGZpbGwgPSBsbG1fbGFiZWwpKSArDQogICAgZ2dwbG90Mjo6Z2VvbV9iYXIoc3RhdCA9ICJpZGVudGl0eSIsIHBvc2l0aW9uID0gImRvZGdlIikgKw0KICAgIGdncGxvdDI6Omdlb21fdGV4dCggZ2dwbG90Mjo6YWVzKGxhYmVsID0gc3ByaW50ZigiJS4yZiUlIiwgcGVyY2VudCkpLCANCiAgICAgICAgICAgICAgICAgICAgICAgIHZqdXN0ID0gLTAuNSwgc2l6ZSA9IDMsIGZvbnRmYWNlID0gImJvbGQiKSArDQogICAgZ2dwbG90Mjo6c2NhbGVfZmlsbF9tYW51YWwodmFsdWVzID0gbGFiZWxfY29sb3JzKSArDQogICAgZ2dwbG90Mjo6c2NhbGVfeV9jb250aW51b3VzKA0KICAgICAgbGltaXRzID0gYygwLCAxMTApLA0KICAgICAgYnJlYWtzID0gc2VxKDAsIDEwMCwgNTApLA0KICAgICAgbGFiZWxzID0gc2NhbGVzOjpsYWJlbF9wZXJjZW50KHNjYWxlID0gMSkNCiAgICApICsNCiAgICBnZ3Bsb3QyOjpnZ3RpdGxlKG1vZGVsX3Nob3J0KSArDQogICAgZ2dwbG90Mjo6dGhlbWVfY2xhc3NpYygpICsNCiAgICBnZ3Bsb3QyOjp0aGVtZSgNCiAgICAgIHBsb3QudGl0bGUgPSAgZ2dwbG90Mjo6ZWxlbWVudF90ZXh0KGNvbG9yID0gY29sb3IsIGhqdXN0ID0gMC41LCBmYWNlID0gImJvbGQiLCBzaXplID0gOCksDQogICAgICBheGlzLnRpdGxlLnggPSAgZ2dwbG90Mjo6ZWxlbWVudF9ibGFuaygpLA0KICAgICAgYXhpcy50aXRsZS55ID0gIGdncGxvdDI6OmVsZW1lbnRfYmxhbmsoKSwNCiAgICAgIGF4aXMudGV4dCA9ICBnZ3Bsb3QyOjplbGVtZW50X3RleHQoc2l6ZSA9IDgsIGZhY2UgPSAnYm9sZCcpLA0KICAgICAgbGVnZW5kLnBvc2l0aW9uID0gIm5vbmUiDQogICAgKQ0KfSkNCg0KIyBDb21iaW5lIHBsb3RzIHVzaW5nIHBhdGNod29yaw0KcGF0Y2h3b3JrOjp3cmFwX3Bsb3RzKHBsb3RzLCBuY29sID0gMiwgYXhpc190aXRsZXMgPSdjb2xsZWN0JykgKw0KICBwYXRjaHdvcms6OnBsb3RfYW5ub3RhdGlvbigNCiAgICB0aXRsZSA9ICJEaXN0cmlidXRpb24gb2YgTExNIExhYmVscyBmb3IgQmluYXJ5IENsYXNzaWZpY2F0aW9uIiwNCiAgICBzdWJ0aXRsZSA9ICI8c3BhbiBzdHlsZT0nY29sb3I6IzgwODA4MCc+Q2hlYXBlcjwvc3Bhbj4gdnMuIDxzcGFuIHN0eWxlPSdjb2xvcjojQzMxNDJEJz5tb3JlIGV4cGVuc2l2ZSAodGltZSwgY29zdCkgPC9zcGFuPiBMTE1zIGJ5IGNvbXBhbnkiLA0KICAgIHRoZW1lID0gZ2dwbG90Mjo6dGhlbWUoDQogICAgICBwbG90LnRpdGxlID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGhqdXN0ID0gMC41LCBmYWNlID0gImJvbGQiLCBzaXplID0gMTEpLA0KICAgICAgcGxvdC5zdWJ0aXRsZSA9IGdndGV4dDo6ZWxlbWVudF9tYXJrZG93bihoanVzdCA9IDAuNSwgZmFjZSA9ICJib2xkIiwgc2l6ZSA9IDEwKSwNCiAgICAgIGF4aXMudGl0bGUueCA9IGdndGV4dDo6ZWxlbWVudF9tYXJrZG93bihoanVzdCA9IDAuNSwgZmFjZSA9ICJib2xkIiwgc2l6ZSA9IDkpLA0KICAgICAgYXhpcy50aXRsZS55ID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGhqdXN0ID0gMC41LCBmYWNlID0gImJvbGQiLCBzaXplID0gOSksDQogICAgICBheGlzLnRleHQueCA9IGdndGV4dDo6ZWxlbWVudF9tYXJrZG93bihoanVzdCA9IDAuNSwgc2l6ZSA9IDcpLA0KICAgICAgYXhpcy50ZXh0LnkgPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oaGp1c3QgPSAxLCBzaXplID0gNyksDQogICAgICBzdHJpcC50ZXh0ID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGZhY2UgPSAiYm9sZCIsIHNpemUgPSA4KSwNCiAgICAgIHBhbmVsLmdyaWQubWFqb3IgPSBnZ3Bsb3QyOjplbGVtZW50X2JsYW5rKCksDQogICAgICBwYW5lbC5ncmlkLm1pbm9yID0gZ2dwbG90Mjo6ZWxlbWVudF9ibGFuaygpLA0KICAgICAgbGVnZW5kLnBvc2l0aW9uID0gIm5vbmUiLA0KICAgICkNCiAgKQ0KYGBgDQoNCg0KIyMgUmVsaWFiaWxpdHkgQXNzZXNzbWVudA0KDQpJbiB0aGlzIHNlY3Rpb24sIHdlIGNvbmR1Y3QgdGhlIHJlbGlhYmlsaXR5IGFzc2Vzc21lbnQuIA0KDQojIyMgRGlzdHJpYnV0aW9uIG9mIE5BLVBlbmFsaXplZCBBZ3JlZW1lbnQgQWNyb3NzIEFydGljbGVzIGJ5IExMTQ0KDQpUaGUgZmlndXJlIGJlbG93IHNob3dzIHRoZSBOQS1wZW5hbGl6ZWQgYWdyZWVtZW50IGRpc3RyaWJ1dGlvbnMgZm9yIGVhY2ggbW9kZWwsIGhpZ2hsaWdodGluZyB0aGUgcGVyY2VudGFnZSBvZiBhcnRpY2xlcyB3aGVyZSBhbGwgZml2ZSByZXBsaWNhdGVzIGdlbmVyYXRlZCBpZGVudGljYWwgY2xhc3NpZmljYXRpb25zLiBUaGUgTkEtcGVuYWx0eSByZWZsZWN0cyBob3cgd2UgY291bnQgYWdyZWVtZW50IGluIHRoZSBwcmVzZW5jZSBvZiBpbnZhbGlkIHJlc3BvbnNlcywgd2hlcmUgTkFzIGFyZSBpZ25vcmVkIGluIHRoZSBudW1lcmF0b3IgYnV0IGFyZSBrZXB0IGluIHRoZSBkZW5vbWluYXRvci4gDQoNCg0KYGBge3IgZGlzdHJpYnV0aW9uX05BX3BlbmFsaXplZCwgY2xhc3Muc291cmNlID0gJ2ZvbGQtaGlkZScsIG1lc3NhZ2U9RkFMU0UsIGZpZy5hbGlnbj0nY2VudGVyJywgZmlnLndpZHRoPTYsIGZpZy5oZWlnaHQ9OCwgZmlnLmZ1bGx3aWR0aD1UUlVFfQ0KIyBsaWJyYXJ5KHRpZHl2ZXJzZSkNCm1pYW1pcmVkID0gJyNDMzE0MkQnDQoNCmNoYXRfbW9kZWxfY29sb3JzID0gcmVwKGMoJyM4MDgwODAnLCAnI0MzMTQyRCcpLCA3KQ0KDQptb2RlX2ZyZXF1ZW5jeSA9IGZ1bmN0aW9uKHgsIGRyb3BfbmE9Rikgew0KICAjIHJldHVybiAwIGlmIGFsbCB2YWx1ZXMgYXJlIE5BDQogIGlmIChhbGwoaXMubmEoeCkpKSB7DQogICAgcmV0dXJuKDApDQogIH0NCiAgIyBjYWxjdWxhdGUgdGhlIGZyZXF1ZW5jeSB0YWJsZQ0KICBmcmVxX3RhYmxlID0gdGFibGUoeCkNCiAgaWYgKGRyb3BfbmE9PUYpew0KICAgICMgcmV0dXJuIHRoZSBmcmVxdWVuY3kgb2YgdGhlIG1vc3QgY29tbW9uIHZhbHVlDQogICAgcmV0dXJuKG1heChmcmVxX3RhYmxlKSAvIGxlbmd0aCh4KSkNCiAgfWVsc2V7DQogICAgcmV0dXJuKG1heChmcmVxX3RhYmxlKS9zdW0oZnJlcV90YWJsZSkpDQogIH0NCn0NCg0KDQojICogR2V0dGluZyB0aGUgRGF0YSAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCg0KYmluYXJ5X2NsYXNzaWZpY2F0aW9uX3Jlc3VsdHMgPSANCiAgcmVhZHI6OnJlYWRfY3N2KCIuLi9yZXN1bHRzL2JpbmFyeV9hbmFseXNpc19kZi5jc3YiKSB8PiANCiAgZHBseXI6OnNlbGVjdCgNCiAgICBkYXRlLCB0aXRsZSwgdGlja2VyczpyZXBfNQ0KICApIHw+IA0KICB0aWR5cjo6cGl2b3RfbG9uZ2VyKA0KICAgIGNvbHMgPSByZXBfMTpyZXBfNSwNCiAgICBuYW1lc190byA9ICJyZXAiLA0KICAgIHZhbHVlc190byA9ICJsbG1fbGFiZWwiDQogICkgfD4gIA0KICBkcGx5cjo6ZmlsdGVyKGNoYXRfbW9kZWwgIT0gJ2V4YW9uZS1kZWVwOjIuNGInKSANCg0KDQoNCiMgKiBTdHVmZiBOZWVkZWQgZm9yIENvbG9yaW5nIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KDQojICogQ2hhdCBNb2RlbHMgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCg0KIyBBc3NpZ24gYWx0ZXJuYXRpbmcgY29sb3JzIHRvIGNoYXRfbW9kZWwgbGFiZWxzDQpjaGF0X21vZGVscyA9IGMoDQogICJwaGk0LW1pbmkiLA0KICAicGhpNDpsYXRlc3QiLA0KICAibGxhbWEzLjI6MUIiLA0KICAibGxhbWEzLjI6M0IiLCANCiAgImdlbW1hMzoxQiIsDQogICJnZW1tYTM6MjdCIiwNCiAgImRlZXBzZWVrLXIxOjEuNUIiLA0KICAiZGVlcHNlZWstcjE6N0IiLA0KICAiY29tbWFuZC1yN2IiLA0KICAiY29tbWFuZC1yLXBsdXMtMDgtMjAyNCIsDQogICJncHQtNG8tbWluaS0yMDI0LTA3LTE4IiwNCiAgImdwdC00by0yMDI0LTExLTIwIiwNCiAgImNsYXVkZS0zLTUtaGFpa3UtMjAyNDEwMjIiLA0KICAiY2xhdWRlLTMtNy1zb25uZXQtMjAyNTAyMTkiDQopDQoNCm1vZGVsX2FiYnJldiA9IGMoDQogICJwaGk0LW1pbmkiICAgICAgICAgICAgICAgID0gInBoaTQtbWluaSIsDQogICJwaGk0OmxhdGVzdCIgICAgICAgICAgICAgID0gInBoaTQ6bGF0ZXN0IiwNCiAgImxsYW1hMy4yOjFCIiAgICAgICAgICAgICAgID0gImxsYW1hMy4yOjFCIiwNCiAgImxsYW1hMy4yOjNCIiAgICAgICAgICAgICAgID0gImxsYW1hMy4yOjNCIiwNCiAgImdlbW1hMzoxQiIgICAgICAgICAgICAgICAgID0gImdlbW1hMzoxQiIsDQogICJnZW1tYTM6MjdCIiAgICAgICAgICAgICAgICA9ICJnZW1tYTM6MjdCIiwNCiAgImRlZXBzZWVrLXIxOjEuNUIiICAgICAgICAgID0gImRlZXBzZWVrLXIxOjEuNUIiLA0KICAiZGVlcHNlZWstcjE6N0IiICAgICAgICAgICAgPSAiZGVlcHNlZWstcjE6N0IiLA0KICAiY29tbWFuZC1yN2IiICAgICAgICAgICAgICAgPSAiY29tbWFuZC1yN2IiLA0KICAiY29tbWFuZC1yLXBsdXMtMDgtMjAyNCIgICAgPSAiY29tbWFuZC1yLXBsdXMiLA0KICAiZ3B0LTRvLW1pbmktMjAyNC0wNy0xOCIgICAgPSAiZ3B0LTRvLW1pbmkiLA0KICAiZ3B0LTRvLTIwMjQtMTEtMjAiICAgICAgICAgPSAiZ3B0LTRvIiwNCiAgImNsYXVkZS0zLTUtaGFpa3UtMjAyNDEwMjIiID0gImNsYXVkZS0zLTUtaGFpa3UiLA0KICAiY2xhdWRlLTMtNy1zb25uZXQtMjAyNTAyMTkiPSAiY2xhdWRlLTMtNy1zb25uZXQiDQopDQoNCg0KIyAqIFBlciBUYXNrIGFuZCBNb2RlbCBMYWJlbHMgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQoNCmJpbmFyeV9jbGFzc2lmaWNhdGlvbl9yZXN1bHRzIHw+ICANCiAgdGlkeXI6OnBpdm90X3dpZGVyKA0KICAgIG5hbWVzX2Zyb20gPSByZXAsDQogICAgdmFsdWVzX2Zyb20gPSBsbG1fbGFiZWwNCiAgKSB8PiANCiAgZHBseXI6OnNlbGVjdCgtZGF0ZSkgLT4gYmluYXJ5X2NsYXNzaWZpY2F0aW9uX3Jlc3VsdHNfd2lkZQ0KDQpiaW5hcnlfYW5hbHlzaXNfZGYgPSANCiAgYmluYXJ5X2NsYXNzaWZpY2F0aW9uX3Jlc3VsdHNfd2lkZSB8Pg0KICBkcGx5cjo6cm93d2lzZSgpIHw+DQogIGRwbHlyOjptdXRhdGUoDQogICAgcGVyY2VudF9hZ3JlZW1lbnQgPSAxMDAqIG1vZGVfZnJlcXVlbmN5KGRwbHlyOjpjX2Fjcm9zcyhzdGFydHNfd2l0aCgicmVwXyIpKSkNCiAgKSB8PiANCiAgZHBseXI6OnVuZ3JvdXAoKQ0KDQphZ3JlZW1lbnQgPSANCiAgYmluYXJ5X2FuYWx5c2lzX2RmIHw+IA0KICBkcGx5cjo6Z3JvdXBfYnkoY2hhdF9tb2RlbCwgcGVyY2VudF9hZ3JlZW1lbnQpIHw+IA0KICBkcGx5cjo6c3VtbWFyaXplKGNvdW50ID0gZHBseXI6Om4oKSkgfD4NCiAgZHBseXI6OnVuZ3JvdXAoKSB8PiANCiAgZHBseXI6Omdyb3VwX2J5KGNoYXRfbW9kZWwpIHw+DQogIGRwbHlyOjptdXRhdGUoDQogICAgcGVyY2VudCA9IDEwMCAqIChjb3VudCAvIHN1bShjb3VudCkpLA0KICAgIGNoYXRfbW9kZWwgPSBtb2RlbF9hYmJyZXZbY2hhdF9tb2RlbF0gfD4gYXMuZmFjdG9yKCksDQogICAgY2hhdF9tb2RlbCA9IGZvcmNhdHM6OmZjdF9yZWxldmVsKA0KICAgICAgY2hhdF9tb2RlbCwgDQogICAgICAicGhpNC1taW5pIiwgInBoaTQ6bGF0ZXN0IiwgImxsYW1hMy4yOjFCIiwgImxsYW1hMy4yOjNCIiwgDQogICAgICAiZ2VtbWEzOjFCIiwgImdlbW1hMzoyN0IiLCAiZGVlcHNlZWstcjE6MS41QiIsICJkZWVwc2Vlay1yMTo3QiIsIA0KICAgICAgImNvbW1hbmQtcjdiIiwgImNvbW1hbmQtci1wbHVzIiwgImdwdC00by1taW5pIiwgImdwdC00byIsIA0KICAgICAgImNsYXVkZS0zLTUtaGFpa3UiLCAiY2xhdWRlLTMtNy1zb25uZXQiDQogICAgKQ0KICApIHw+DQogIGRwbHlyOjp1bmdyb3VwKCkNCg0KDQojICogUGxvdCBMaXN0IC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCg0KY2hhdF9tb2RlbF9jb2xvcnMgPSBzZXROYW1lcyhjaGF0X21vZGVsX2NvbG9ycywgdW5pcXVlKGFncmVlbWVudCRjaGF0X21vZGVsKSApDQoNCiMgU3BsaXQgZGF0YSBieSBjaGF0X21vZGVsIGFuZCBnZXQgdGhlIG5hbWVzIGZvciB0aXRsZXMNCmNoYXRfbW9kZWxfZ3JvdXBzID0gYWdyZWVtZW50IHw+IA0KICBkcGx5cjo6Z3JvdXBfYnkoY2hhdF9tb2RlbCkgfD4gDQogIGRwbHlyOjpncm91cF9zcGxpdCgpDQoNCmNoYXRfbW9kZWxfbmFtZXMgPSBhZ3JlZW1lbnQgfD4gDQogIGRwbHlyOjpncm91cF9ieShjaGF0X21vZGVsKSB8PiANCiAgZHBseXI6Omdyb3VwX2tleXMoKSB8PiANCiAgZHBseXI6OnB1bGwoY2hhdF9tb2RlbCkNCg0KIyBDcmVhdGUgYSBsaXN0IG9mIHBsb3RzIHdpdGggdGhlIGNoYXRfbW9kZWwgYXMgdGhlIHRpdGxlDQpwbG90X2xpc3QgPSBwdXJycjo6bWFwMihjaGF0X21vZGVsX2dyb3VwcywgY2hhdF9tb2RlbF9uYW1lcywgfnsNCiAgdGl0bGVfY29sb3IgPSBjaGF0X21vZGVsX2NvbG9yc1tbLnldXQ0KICB0aXRsZV9odG1sID0gZ2x1ZTo6Z2x1ZSgiPHNwYW4gc3R5bGU9J2NvbG9yOnt0aXRsZV9jb2xvcn0nPnsueX08L3NwYW4+IikNCiAgDQogIGdncGxvdDI6OmdncGxvdCgueCwgZ2dwbG90Mjo6YWVzKHggPSBwZXJjZW50X2FncmVlbWVudCwgeSA9IHBlcmNlbnQsIGZpbGwgPSBjaGF0X21vZGVsKSkgKw0KICAgIGdncGxvdDI6Omdlb21fYmFyKHN0YXQgPSAnaWRlbnRpdHknLCBjb2xvciA9ICdibGFjaycpICsNCiAgICBnZ3Bsb3QyOjpzY2FsZV9maWxsX21hbnVhbCh2YWx1ZXMgPSBjaGF0X21vZGVsX2NvbG9ycykgKw0KICAgIGdncGxvdDI6OmxhYnMoDQogICAgICB0aXRsZSA9IHRpdGxlX2h0bWwsDQogICAgICB4ID0gIlBlcmNlbnQgQWdyZWVtZW50IHdpdGhpbiBhIExMTSAoTkEgUGVuYWx0eSkiLA0KICAgICAgI3kgPSAnUGVyY2VudGFnZSBvZiBBcnRpY2xlcycsDQogICAgICB5ID0gJycsDQogICAgICBmaWxsID0gIkxMTSBNb2RlbCINCiAgICApICsNCiAgICBnZ3Bsb3QyOjpzY2FsZV94X2NvbnRpbnVvdXMoYnJlYWtzID0gc2VxKDAsIDEwMCwgMjApKSArDQogICAgZ2dwbG90Mjo6c2NhbGVfeV9jb250aW51b3VzKA0KICAgICAgbGFiZWxzID0gc2NhbGVzOjpwZXJjZW50X2Zvcm1hdChzY2FsZSA9IDEpLA0KICAgICAgbGltaXRzID0gYygwLCAxNjApLA0KICAgICAgYnJlYWtzID0gc2VxKDAsIDEwMCwgNTApDQogICAgKSArDQogICAgZ2dwbG90Mjo6Z2VvbV90ZXh0KA0KICAgICAgZ2dwbG90Mjo6YWVzKGxhYmVsID0gc3ByaW50ZigiJS4xZiUlIiwgcGVyY2VudCkpLA0KICAgICAgdmp1c3QgPSAtMC41LA0KICAgICAgc2l6ZSA9IDIuNSwNCiAgICAgIGZvbnRmYWNlID0gImJvbGQiDQogICAgKSArDQogICAgZ2dwbG90Mjo6dGhlbWVfY2xhc3NpYygpICsNCiAgICBnZ3Bsb3QyOjp0aGVtZSgNCiAgICAgIHBsb3QudGl0bGUgPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oaGp1c3QgPSAwLjUsIGZhY2UgPSAiYm9sZCIsIHNpemUgPSA5KSwNCiAgICAgIGF4aXMudGl0bGUueCA9IGdndGV4dDo6ZWxlbWVudF9tYXJrZG93bihoanVzdCA9IDAuNSwgZmFjZSA9ICJib2xkIiwgc2l6ZSA9IDkpLA0KICAgICAgYXhpcy50aXRsZS55ID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGhqdXN0ID0gMC41LCBmYWNlID0gImJvbGQiLCBzaXplID0gOSksDQogICAgICBheGlzLnRleHQueCA9IGdndGV4dDo6ZWxlbWVudF9tYXJrZG93bihoanVzdCA9IDAuNSwgc2l6ZSA9IDcsIGZhY2UgPSAiYm9sZCIpLA0KICAgICAgYXhpcy50ZXh0LnkgPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oaGp1c3QgPSAxLCBzaXplID0gNywgZmFjZSA9ICJib2xkIiksDQogICAgICBzdHJpcC50ZXh0ID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGZhY2UgPSAiYm9sZCIsIHNpemUgPSA4KSwNCiAgICAgIHBhbmVsLmdyaWQubWFqb3IgPSBnZ3Bsb3QyOjplbGVtZW50X2JsYW5rKCksDQogICAgICBwYW5lbC5ncmlkLm1pbm9yID0gZ2dwbG90Mjo6ZWxlbWVudF9ibGFuaygpLA0KICAgICAgbGVnZW5kLnBvc2l0aW9uID0gIm5vbmUiDQogICAgKQ0KfSkNCg0KDQpwYXRjaHdvcms6OndyYXBfcGxvdHMocGxvdF9saXN0LCBuY29sID0gMiwgYXhpc190aXRsZXMgPSdjb2xsZWN0JywgYXhlcyA9ICdrZWVwJykgKw0KICBwYXRjaHdvcms6OnBsb3RfYW5ub3RhdGlvbigNCiAgICB0aXRsZSA9ICJEaXN0cmlidXRpb24gb2YgTkEtUGVuYWxpemVkIEFncmVlbWVudCBBY3Jvc3MgQXJ0aWNsZXMgYnkgTExNIiwNCiAgICBzdWJ0aXRsZSA9ICI8c3BhbiBzdHlsZT0nY29sb3I6IzgwODA4MCc+Q2hlYXBlcjwvc3Bhbj4gdnMuIDxzcGFuIHN0eWxlPSdjb2xvcjojQzMxNDJEJz5tb3JlIGV4cGVuc2l2ZSAodGltZSwgY29zdCkgPC9zcGFuPiBMTE1zIGJ5IGNvbXBhbnkiLA0KICAgIHRoZW1lID0gZ2dwbG90Mjo6dGhlbWUoDQogICAgICBwbG90LnRpdGxlID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGhqdXN0ID0gMC41LCBmYWNlID0gImJvbGQiLCBzaXplID0gOSksDQogICAgICBwbG90LnN1YnRpdGxlID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGhqdXN0ID0gMC41LCBmYWNlID0gImJvbGQiLCBzaXplID0gOSksDQogICAgICBheGlzLnRpdGxlLnggPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oaGp1c3QgPSAwLjUsIGZhY2UgPSAiYm9sZCIsIHNpemUgPSA5KSwNCiAgICAgIGF4aXMudGl0bGUueSA9IGdndGV4dDo6ZWxlbWVudF9tYXJrZG93bihoanVzdCA9IDAuNSwgZmFjZSA9ICJib2xkIiwgc2l6ZSA9IDkpLA0KICAgICAgYXhpcy50ZXh0LnggPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oaGp1c3QgPSAwLjUsIHNpemUgPSA3KSwNCiAgICAgIGF4aXMudGV4dC55ID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGhqdXN0ID0gMSwgc2l6ZSA9IDcpLA0KICAgICAgc3RyaXAudGV4dCA9IGdndGV4dDo6ZWxlbWVudF9tYXJrZG93bihmYWNlID0gImJvbGQiLCBzaXplID0gOCksDQogICAgICBwYW5lbC5ncmlkLm1ham9yID0gZ2dwbG90Mjo6ZWxlbWVudF9ibGFuaygpLA0KICAgICAgcGFuZWwuZ3JpZC5taW5vciA9IGdncGxvdDI6OmVsZW1lbnRfYmxhbmsoKSwNCiAgICAgIGxlZ2VuZC5wb3NpdGlvbiA9ICJub25lIiwNCiAgICApDQogICkgLT4gY29tYmluZWRfcGxvdA0KDQpjb3dwbG90OjpnZ2RyYXcoKSArDQogIGNvd3Bsb3Q6OmRyYXdfbGFiZWwoDQogICAgIlBlcmNlbnRhZ2Ugb2YgQXJ0aWNsZXMiLA0KICAgIHggPSAwLjAyLCB5ID0gMC41LCBhbmdsZSA9IDkwLCB2anVzdCA9IDAuNSwNCiAgICBmb250ZmFjZSA9ICJib2xkIiwgc2l6ZSA9IDkNCiAgKSArDQogIGNvd3Bsb3Q6OmRyYXdfcGxvdChjb21iaW5lZF9wbG90LCB4ID0gMC4wNSwgeSA9IDAsIHdpZHRoID0gMC45NSwgaGVpZ2h0ID0gMSkNCg0KDQpgYGANCg0KDQojIyMgQ29tcHV0aW5nIHRoZSBSZWxpYWJpbGl0eSBNZXRyaWNzIHsjY29tcHV0ZV9yZWxpYWJpbGl0eV9tZXRyaWNzX2Jpbn0NCg0KDQpgYGB7ciBiaW5hcnlfcmVsaWFiaWxpdHlfbWV0cmljcywgcmVzdWx0cz0nYXNpcycsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0V9DQpwYWNtYW46OnBfbG9hZChpcnJDQUMsIHN0cmluZ3IsIHRpZHl2ZXJzZSkNCmJpbmFyeV9hbmFseXNpc19kZiA8LSByZWFkcjo6cmVhZF9jc3YoIi4uL3Jlc3VsdHMvYmluYXJ5X2FuYWx5c2lzX2RmLmNzdiIpDQoNCiMgYmluYXJ5IGNsYXNzaWZpY2F0aW9uIHJlbGlhYmlsaXR5IG1ldHJpY3MNCmRmID0gcmVsaWFiaWxpdHlfY29lZnMoYmluYXJ5X2FuYWx5c2lzX2RmLCA2OjEwKQ0KZGYgPSBkZiB8Pg0KICBtdXRhdGUoYWNyb3NzKDM6NiwgfnJvdW5kKC54LDMpKSkNCg0KY2F0KCJUaGUgcmVsaWFiaWxpdHkgbWV0cmljcyBmb3IgdGhlIGJpbmFyeSBjbGFzc2lmaWNhdGlvbiBleHBlcmltZW50IGFyZSBhcyBmb2xsb3dzOlxuIikNCkRUOjpkYXRhdGFibGUoZGYsIHJvd25hbWVzID0gRkFMU0UsIG9wdGlvbnMgPSBsaXN0KHBhZ2VMZW5ndGggPSAxMCwgc2Nyb2xsWCA9IFRSVUUpKQ0KDQpjYXQoIlRoZSBtZWFuIHBlcmNlbnQgYWdyZWVtZW50IHdpdGggTkEgcGVuYWx0eSBmb3IgZWFjaCBtb2RlbCBpcyBhcyBmb2xsb3dzOlxuIikNCnBhX3RhYmxlID0gcGFfc3VtbWFyeShiaW5hcnlfYW5hbHlzaXNfZGYsICJwZXJjZW50X2FncmVlbWVudCIsIGRpZ2l0cyA9IDMpIHw+IA0KICB0aWR5cjo6c2VwYXJhdGUoDQogICAgbVBhLCBpbnRvID0gYygibWVhbl9wZXJjZW50X2FncmVlbWVudCIsICJzZF9wZXJjZW50X2FncmVlbWVudCIpLCANCiAgICBzZXAgPSAiIFxcKCINCiAgKSB8Pg0KICBkcGx5cjo6bXV0YXRlKA0KICAgIG1lYW5fcGVyY2VudF9hZ3JlZW1lbnQgPSBhcy5udW1lcmljKG1lYW5fcGVyY2VudF9hZ3JlZW1lbnQpLA0KICAgIHNkX3BlcmNlbnRfYWdyZWVtZW50ID0gDQogICAgICBhcy5udW1lcmljKHN0cmluZ3I6OnN0cl9yZW1vdmUoc2RfcGVyY2VudF9hZ3JlZW1lbnQsICJcXCkiKSkNCiAgKQ0KDQpEVDo6ZGF0YXRhYmxlKHBhX3RhYmxlLCByb3duYW1lcyA9IEZBTFNFLCANCiAgICAgICAgICAgICAgb3B0aW9ucyA9IGxpc3QocGFnZUxlbmd0aCA9IDE0LCBzY3JvbGxYID0gVFJVRSkNCikgDQoNCmNhdCgNCiAgcGFzdGUoDQogICAgJ0Zyb20gdGhlIHRhYmxlIGFib3ZlLCB3ZSBjYW4gc2VlIHRoYXQgdGhlIHN0YW5kYXJkIGRldmlhdGlvbiBvZiB0aGUgcGVyY2VudCBhZ3JlZW1lbnQgaXMgcXVpdGUgbG93IGZvciwnLA0KICAgIHBhc3RlKHBhX3RhYmxlJG1vZGVsW3doaWNoKHBhX3RhYmxlJHNkX3BlcmNlbnRfYWdyZWVtZW50IDwgMC4xKV0sIGNvbGxhcHNlID0gJywgJyksDQogICAgJ2luZGljYXRpbmcgYSBoaWdoIGxldmVsIG9mIGFncmVlbWVudCBhbW9uZyB0aG9zZSBMTE1zIGluIHRoZSBiaW5hcnkgY2xhc3NpZmljYXRpb24gZXhwZXJpbWVudC4gSG93ZXZlciwgdGhlIG90aGVycyBtb2RlbHMgaGF2ZSBoaWdoZXIgc3RhbmRhcmQgZGV2aWF0aW9ucywgZXNwZWNpYWxseSBmb3IgdGhlJywNCiAgICBwYXN0ZShwYV90YWJsZSRtb2RlbFt3aGljaChwYV90YWJsZSRzZF9wZXJjZW50X2FncmVlbWVudCA+IDAuMTUpXSwgY29sbGFwc2UgPSAnLCAnKSwNCiAgICAnbW9kZWwuIFxuJykNCikNCg0KcGFfdGFibGVfZHJvcF9uYSA9IHBhX3N1bW1hcnkoYmluYXJ5X2FuYWx5c2lzX2RmLCAicGVyY2VudF9hZ3JlZW1lbnRfZHJvcF9uYSIsIGRpZ2l0cyA9IDMpIHw+IA0KICB0aWR5cjo6c2VwYXJhdGUoDQogICAgbVBhLCBpbnRvID0gYygibWVhbl9wZXJjZW50X2FncmVlbWVudF9kcm9wX25hIiwgInNkX3BlcmNlbnRfYWdyZWVtZW50IiksIA0KICAgIHNlcCA9ICIgXFwoIg0KICApIHw+DQogIGRwbHlyOjptdXRhdGUoDQogICAgbWVhbl9wZXJjZW50X2FncmVlbWVudF9kcm9wX25hID0gYXMubnVtZXJpYyhtZWFuX3BlcmNlbnRfYWdyZWVtZW50X2Ryb3BfbmEpLA0KICAgIHNkX3BlcmNlbnRfYWdyZWVtZW50ID0gDQogICAgICBhcy5udW1lcmljKHN0cmluZ3I6OnN0cl9yZW1vdmUoc2RfcGVyY2VudF9hZ3JlZW1lbnQsICJcXCkiKSkNCiAgKQ0KDQpkZl93aWRlID0gZGYgfD4gDQogIGRwbHlyOjpzZWxlY3QobW9kZWwsIGNvZWZmLm5hbWUsIGNvZWZmLnZhbCkgfD4gDQogIHRpZHlyOjpwaXZvdF93aWRlcihuYW1lc19mcm9tID0gY29lZmYubmFtZSwgdmFsdWVzX2Zyb20gPSBjb2VmZi52YWwpDQoNCmRmX3dpZGUgPSBkcGx5cjo6bGVmdF9qb2luKA0KICB4ID0gcGFfdGFibGUgfD4gZHBseXI6OnNlbGVjdCgtc2RfcGVyY2VudF9hZ3JlZW1lbnQpLCB5ID0gZGZfd2lkZSwgDQogIGJ5ID0gIm1vZGVsIg0KKSANCg0KZGZfd2lkZSA9IGRwbHlyOjpsZWZ0X2pvaW4oDQogIHggPSBwYV90YWJsZV9kcm9wX25hIHw+IGRwbHlyOjpzZWxlY3QoLXNkX3BlcmNlbnRfYWdyZWVtZW50KSwgeSA9IGRmX3dpZGUsIA0KICBieSA9ICJtb2RlbCIgIA0KKSB8PiANCiAgZHBseXI6OnJlbmFtZSgNCiAgICBgUGVyY2VudCBBZ3JlZW1lbnQgKHdpdGggTkEgUGVuYWx0eSlgID0gbWVhbl9wZXJjZW50X2FncmVlbWVudCwNCiAgICBgUGVyY2VudCBBZ3JlZW1lbnQgKElnbm9yaW5nIE5BKWAgPSBtZWFuX3BlcmNlbnRfYWdyZWVtZW50X2Ryb3BfbmENCiAgKSB8PiANCiAgZHBseXI6OnNlbGVjdCgNCiAgICBtb2RlbCwgDQogICAgYFBlcmNlbnQgQWdyZWVtZW50ICh3aXRoIE5BIFBlbmFsdHkpYCwgYFBlcmNlbnQgQWdyZWVtZW50IChJZ25vcmluZyBOQSlgLCANCiAgICBkcGx5cjo6ZXZlcnl0aGluZygpIA0KICApDQoNCiMgZ2V0IHRoZSBtb2RlbHMgd2l0aCBoaWdoIHN0YW5kYXJkIGRldmlhdGlvbg0KaGlnaF9wYV9zZF9tb2RlbHNfaW5kZXggPSB3aGljaChwYV90YWJsZSRzZF9wZXJjZW50X2FncmVlbWVudCA+IDAuMTUpDQoNCiMgYSBzdW1tYXJpemVkIHRhYmxlIG9mIHRoZSBtZXRyaWNzDQpkZl93aWRlID0gZGZfd2lkZSB8PiANCiAgIyByb3VuZGluZyBudW1lcmljIGNvbHVtbnMgdG8gMyBkZWNpbWFsIHBsYWNlcw0KICBkcGx5cjo6bXV0YXRlKA0KICAgIGRwbHlyOjphY3Jvc3MoZHBseXI6OndoZXJlKGlzLm51bWVyaWMpLCB+IHJvdW5kKC4sIDMpKSwNCiAgICBoaWdoX3BhX3NkID0gaWZlbHNlKA0KICAgICAgbW9kZWwgJWluJSBwYV90YWJsZSRtb2RlbFtwYV90YWJsZSRzZF9wZXJjZW50X2FncmVlbWVudCA+IDAuMTVdLA0KICAgICAgVFJVRSwgRkFMU0UNCiAgICApLA0KICAgIGBBUEkgQ29zdHMgLyAxTSBJbnB1dCBUb2tlbnNgID0gYygwLjgsIDMsIDIuNSwgMCwgMCwgMCwgMCwgMCwgMi41LCAwLjE1LCAwLCAwLCAwLCAwKSwNCiAgICBgQVBJIENvc3RzIC8gMU0gT3V0cHV0IFRva2Vuc2A9IGMoNCwgIDE1LCAgMTAsIDAsIDAgLDAsIDAsIDAsICAxMCwgIDAuNiwgMCwgMCwgMCwgMCkNCiAgKQ0KDQojIHNhdmUgdGhlIHJlbGlhYmlsaXR5IG1ldHJpY3MgZGF0YWZyYW1lIGFzIFJEUyBhbmQgQ1NWIGZpbGVzDQpyZWFkcjo6d3JpdGVfcmRzKGRmX3dpZGUsICIuLi9yZXN1bHRzL2JpbmFyeV9yZWxpYWJpbGl0eV9tZXRyaWNzX21lYW5zLnJkcyIpDQpyZWFkcjo6d3JpdGVfY3N2KGRmX3dpZGUsICIuLi9yZXN1bHRzL2JpbmFyeV9yZWxpYWJpbGl0eV9tZXRyaWNzX21lYW5zLmNzdiIpDQpyZWFkcjo6d3JpdGVfcmRzKHBhX3RhYmxlLCAiLi4vcmVzdWx0cy9iaW5hcnlfcGFfc3VtbWFyeS5yZHMiKQ0KcmVhZHI6OndyaXRlX2NzdihwYV90YWJsZSwgIi4uL3Jlc3VsdHMvYmluYXJ5X3BhX3N1bW1hcnkuY3N2IikNCnJlYWRyOjp3cml0ZV9yZHMoZGYsICIuLi9yZXN1bHRzL2JpbmFyeV9yZWxpYWJpbGl0eV9tZXRyaWNzLnJkcyIpDQpyZWFkcjo6d3JpdGVfY3N2KGRmLCAiLi4vcmVzdWx0cy9iaW5hcnlfcmVsaWFiaWxpdHlfbWV0cmljcy5jc3YiKQ0KYGBgDQoNCg0KVGhlIENTViBmaWxlcyBjb250YWluaW5nIHRoZSByZWxpYWJpbGl0eSBtZXRyaWNzIGZvciB0aGUgYmluYXJ5IGNsYXNzaWZpY2F0aW9uIGV4cGVyaW1lbnQgY2FuIGJlIGFjY2Vzc2VkIGF0IFtiaW5hcnlfcmVsaWFiaWxpdHlfbWV0cmljc19tZWFucy5jc3ZdKGh0dHBzOi8vZ2l0aHViLmNvbS9mbWVnYWhlZC9sbG1fY29uc2lzdGVuY3kvYmxvYi9tYWluL2JpbmFyeV9yZWxpYWJpbGl0eV9tZXRyaWNzX21lYW5zLmNzdiksIFtiaW5hcnlfcGFfc3VtbWFyeS5jc3ZdKGh0dHBzOi8vZ2l0aHViLmNvbS9mbWVnYWhlZC9sbG1fY29uc2lzdGVuY3kvYmxvYi9tYWluL3Jlc3VsdHMvYmluYXJ5X3BhX3N1bW1hcnkuY3N2KSwgYW5kIFtiaW5hcnlfcmVsaWFiaWxpdHlfbWV0cmljcy5jc3ZdKGh0dHBzOi8vZ2l0aHViLmNvbS9mbWVnYWhlZC9sbG1fY29uc2lzdGVuY3kvYmxvYi9tYWluL3Jlc3VsdHMvYmluYXJ5X3JlbGlhYmlsaXR5X21ldHJpY3MuY3N2KS4gDQoNCg0KIyMjIEludHJhLUxMTSBSZWxpYWJpbGl0eQ0KDQpJbiB0aGlzIHNlY3Rpb24sIHdlIHN0dWR5IHRoZSBjaGFuY2UtYWRqdXN0ZWQgcmVsaWFiaWxpdHkgY29lZmZpY2llbnQgZXN0aW1hdGVzIChkb3RzKSBhbmQgdGhlaXIgXHZ7U31pZFwne2F9ay1hZGp1c3RlZCBjb25maWRlbmNlIGludGVydmFscyAod2hpc2tlcnMpLCBjb25zdHJ1Y3RlZCB0byBtYWludGFpbiA5MCUgZmFtaWx5LXdpc2UgY29uZmlkZW5jZSB3aXRoaW4gZWFjaCBjb3N0IGdyb3VwLiANCg0KYGBge3IgaW50cmFfbGxtLCBjbGFzcy5zb3VyY2UgPSAnZm9sZC1oaWRlJywgbWVzc2FnZT1GQUxTRSwgZmlnLmFsaWduPSdjZW50ZXInLCBmaWcud2lkdGg9NiwgZmlnLmhlaWdodD04LCBmaWcuZnVsbHdpZHRoPVRSVUV9DQojIGxpYnJhcnkodGlkeXZlcnNlKQ0KbGlicmFyeShncmlkKQ0KbGlicmFyeShncmlkRXh0cmEpDQoNCiMgLS0tLSBTZXR1cCAtLS0tDQpjaGF0X21vZGVscyA8LSBjKA0KICAicGhpNC1taW5pIiwNCiAgImxsYW1hMy4yOjFCIiwNCiAgImdlbW1hMzoxQiIsICAgICAgICAgICAgICAgIA0KICAiZGVlcHNlZWstcjE6MS41QiIsDQogICJjb21tYW5kLXI3YiIsDQogICJncHQtNG8tbWluaS0yMDI0LTA3LTE4IiwNCiAgImNsYXVkZS0zLTUtaGFpa3UtMjAyNDEwMjIiLA0KICAicGhpNDpsYXRlc3QiLCANCiAgImxsYW1hMy4yOjNCIiwgDQogICJnZW1tYTM6MjdCIiwNCiAgImRlZXBzZWVrLXIxOjdCIiwNCiAgImNvbW1hbmQtci1wbHVzLTA4LTIwMjQiLA0KICAiZ3B0LTRvLTIwMjQtMTEtMjAiLA0KICAiY2xhdWRlLTMtNy1zb25uZXQtMjAyNTAyMTkiDQopDQoNCm1vZGVsX2FiYnJldiA9IGMoDQogICJwaGk0LW1pbmkiICAgICAgICAgICAgICAgICA9ICJwaGk0LW1pbmkiLA0KICAibGxhbWEzLjI6MUIiICAgICAgICAgICAgICA9ICJsbGFtYTMuMjoxQiIsDQogICJnZW1tYTM6MUIiICAgICAgICAgICAgICAgID0gImdlbW1hMzoxQiIsDQogICJkZWVwc2Vlay1yMToxLjVCIiAgICAgICAgID0gImRlZXBzZWVrLXIxOjEuNUIiLA0KICAiY29tbWFuZC1yN2IiICAgICAgICAgICAgICA9ICJjb21tYW5kLXI3YiIsDQogICJncHQtNG8tbWluaS0yMDI0LTA3LTE4IiAgID0gImdwdC00by1taW5pIiwNCiAgImNsYXVkZS0zLTUtaGFpa3UtMjAyNDEwMjIiPSAiY2xhdWRlLTMtNS1oYWlrdSIsDQogICJwaGk0OmxhdGVzdCIgICAgICAgICAgICAgID0gInBoaTQ6bGF0ZXN0IiwNCiAgImxsYW1hMy4yOjNCIiAgICAgICAgICAgICAgPSAibGxhbWEzLjI6M0IiLA0KICAiZ2VtbWEzOjI3QiIgICAgICAgICAgICAgICA9ICJnZW1tYTM6MjdCIiwNCiAgImRlZXBzZWVrLXIxOjdCIiAgICAgICAgICAgPSAiZGVlcHNlZWstcjE6N0IiLA0KICAiY29tbWFuZC1yLXBsdXMtMDgtMjAyNCIgICA9ICJjb21tYW5kLXItcGx1cyIsDQogICJncHQtNG8tMjAyNC0xMS0yMCIgICAgICAgID0gImdwdC00byIsDQogICJjbGF1ZGUtMy03LXNvbm5ldC0yMDI1MDIxOSIgPSAiY2xhdWRlLTMtNy1zb25uZXQiDQopDQoNCmNoYXRfbW9kZWxfY29sb3JzID0gcmVwKGMoJyM4MDgwODAnLCAnI0MzMTQyRCcpLCBlYWNoID0gNykNCmNoYXRfbW9kZWxfY29sb3JzID0gc3RhdHM6OnNldE5hbWVzKGNoYXRfbW9kZWxfY29sb3JzLCBtb2RlbF9hYmJyZXZbY2hhdF9tb2RlbHNdKQ0KDQoNCm1vZGVsX2xhYmVscyA8LSBwdXJycjo6bWFwX2NociggbW9kZWxfYWJicmV2W2NoYXRfbW9kZWxzXSwgfnsNCiAgaWYgKGdyZXBsKCJec3BhY2VyIiwgLngpKSByZXR1cm4oIiIpDQogIGdsdWU6OmdsdWUoIjxzcGFuIHN0eWxlPSdjb2xvcjp7Y2hhdF9tb2RlbF9jb2xvcnNbLnhdfSc+PGI+ey54fTwvYj48L3NwYW4+IikNCn0pDQpuYW1lcyhtb2RlbF9sYWJlbHMpIDwtIG1vZGVsX2FiYnJldltjaGF0X21vZGVsc10NCg0KYWxwaGEgPSAoMS0oMS0uMSleKDEvNykpDQoNCg0KIyAtLS0tIExvYWQgRGF0YSAtLS0tDQppbnRyYV9kZiA9IHJlYWRyOjpyZWFkX2NzdigiLi4vcmVzdWx0cy9iaW5hcnlfcmVsaWFiaWxpdHlfbWV0cmljcy5jc3YiKSB8Pg0KICBkcGx5cjo6bXV0YXRlKA0KICAgIGxvd2VyX2NpID0gY29lZmYudmFsIC0gc3RhdHM6OnFub3JtKDEtYWxwaGEvMikgKiBjb2VmZi5zZSwNCiAgICB1cHBlcl9jaSA9IGNvZWZmLnZhbCArIHN0YXRzOjpxbm9ybSgxLWFscGhhLzIpKiBjb2VmZi5zZSwNCiAgICBhYmJyZXYgPSBmYWN0b3IobW9kZWxfYWJicmV2W21vZGVsXSksDQogICAgYWJicmV2ID0gZm9yY2F0czo6ZmN0X3JlbGV2ZWwoYWJicmV2LCByZXYobW9kZWxfYWJicmV2W2NoYXRfbW9kZWxzXSkgKQ0KICApDQoNCmludHJhX3dpZGUgPSBpbnRyYV9kZiB8PiANCiAgZHBseXI6OnNlbGVjdChhYmJyZXYsIGNvZWZmLm5hbWUsIGNvZWZmLnZhbCkgfD4gDQogIHRpZHlyOjpwaXZvdF93aWRlcihuYW1lc19mcm9tID0gY29lZmYubmFtZSwgdmFsdWVzX2Zyb20gPSBjb2VmZi52YWwpIHw+IA0KICBkcGx5cjo6YXJyYW5nZShkcGx5cjo6ZGVzYyhhYmJyZXYpKSANCmNvbG5hbWVzKGludHJhX3dpZGUpIDwtIGMoIkxMTSIsICJDIiwgIkYiLCAnQUMnLCAiQlAiLCAiSyIpDQoNCg0KaW50cmFfd2lkZSA9IGludHJhX3dpZGUgfD4gDQogIGRwbHlyOjpzZWxlY3QoTExNLCBBQywgQlAsIEMsIEYsIEspIHw+IA0KICBkcGx5cjo6bXV0YXRlKA0KICAgIGRwbHlyOjphY3Jvc3Mod2hlcmUoaXMubnVtZXJpYyksIH4gaWZlbHNlKGlzLm5hKC54KSwgIi0tIiwgc3ByaW50ZigiJS4yZiIsIC54KSkpDQogICkNCg0KDQojICpQbG90IHRoZSBEYXRhIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCg0KaW50cmFfZGYgfD4gDQogIGdncGxvdDI6OmdncGxvdChnZ3Bsb3QyOjphZXMoeCA9IGFiYnJldiwgeSA9IGNvZWZmLnZhbCkpICsNCiAgZ2dwbG90Mjo6Z2VvbV9ibGFuaygpICsNCiAgZ2dwbG90Mjo6Z2VvbV9wb2ludChnZ3Bsb3QyOjphZXMoY29sb3IgPSBhYmJyZXYpLCBzaXplID0gMS41LCBuYS5ybSA9IFRSVUUpICsNCiAgZ2dwbG90Mjo6Z2VvbV9lcnJvcmJhcigNCiAgICBnZ3Bsb3QyOjphZXMoeW1pbiA9IGxvd2VyX2NpLCB5bWF4ID0gdXBwZXJfY2ksIGNvbG9yID0gYWJicmV2KSwNCiAgICB3aWR0aCA9IDAuNSwNCiAgICBuYS5ybSA9IFRSVUUNCiAgKSArDQogICMgQW5ub3RhdGUgbG93ZXIgQ0kgdmFsdWVzDQogIGdncGxvdDI6Omdlb21fdGV4dCgNCiAgICBkYXRhID0gaW50cmFfZGYsDQogICAgZ2dwbG90Mjo6YWVzKA0KICAgICAgeSA9IGxvd2VyX2NpLA0KICAgICAgbGFiZWwgPSBzcHJpbnRmKCIlLjNmIiwgbG93ZXJfY2kpLA0KICAgICAgY29sb3IgPSBhYmJyZXYNCiAgICApLA0KICAgIHNpemUgPSAyLjI1LA0KICAgIGhqdXN0ID0gMS4yLA0KICAgIG5hLnJtID0gVFJVRQ0KICApICsNCiAgDQogICMgQW5ub3RhdGUgdXBwZXIgQ0kgdmFsdWVzDQogIGdncGxvdDI6Omdlb21fdGV4dCgNCiAgICBkYXRhID0gaW50cmFfZGYsDQogICAgZ2dwbG90Mjo6YWVzKA0KICAgICAgeSA9IHVwcGVyX2NpLA0KICAgICAgbGFiZWwgPSBzcHJpbnRmKCIlLjNmIiwgdXBwZXJfY2kpLA0KICAgICAgY29sb3IgPSBhYmJyZXYNCiAgICApLA0KICAgIHNpemUgPSAyLjI1LA0KICAgIGhqdXN0ID0gLTAuMiwNCiAgICBuYS5ybSA9IFRSVUUNCiAgKSArDQogIGdncGxvdDI6OnNjYWxlX2NvbG9yX21hbnVhbCh2YWx1ZXMgPSBjaGF0X21vZGVsX2NvbG9ycywgbmEudHJhbnNsYXRlID0gRkFMU0UpICsNCiAgZ2dwbG90Mjo6c2NhbGVfeF9kaXNjcmV0ZShsYWJlbHMgPSBtb2RlbF9sYWJlbHMpICsNCiAgZ2dwbG90Mjo6c2NhbGVfeV9jb250aW51b3VzKGxpbWl0cyA9IGMoMC43ODUsIDEuMDUzKSwgYnJlYWtzID0gc2VxKDAuOCwgMS4wLCAwLjA1KSkgKw0KICBnZ3Bsb3QyOjpjb29yZF9mbGlwKCkgKw0KICBnZ3Bsb3QyOjpmYWNldF93cmFwKH4gY29lZmYubmFtZSwgbmNvbCA9IDIsIHNjYWxlcyA9ICJmaXhlZCIsIGF4ZXMgPSAnYWxsJykgKw0KICBnZ3Bsb3QyOjpsYWJzKA0KICAgIHRpdGxlID0gIkludHJhLUxMTSBSZWxpYWJpbGl0eSIsDQogICAgc3VidGl0bGUgPSAiPHNwYW4gc3R5bGU9J2NvbG9yOiM4MDgwODAnPkNoZWFwZXI8L3NwYW4+IHZzLiA8c3BhbiBzdHlsZT0nY29sb3I6I0MzMTQyRCc+bW9yZSBleHBlbnNpdmUgKHRpbWUsIGNvc3QpIDwvc3Bhbj4gTExNcyBieSBjb21wYW55IiwNCiAgICB4ID0gTlVMTCwNCiAgICB5ID0gTlVMTA0KICApICsNCiAgZ2dwbG90Mjo6dGhlbWVfbWluaW1hbCgpICsNCiAgZ2dwbG90Mjo6dGhlbWUoDQogICAgcGxvdC50aXRsZSA9IGdndGV4dDo6ZWxlbWVudF9tYXJrZG93bihoanVzdCA9IDAuNSwgZmFjZSA9ICJib2xkIiwgc2l6ZSA9IDkpLA0KICAgIHBsb3Quc3VidGl0bGUgPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oaGp1c3QgPSAwLjUsIGZhY2UgPSAiYm9sZCIsIHNpemUgPSA5KSwNCiAgICBheGlzLnRpdGxlLnggPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oaGp1c3QgPSAwLjUsIGZhY2UgPSAiYm9sZCIsIHNpemUgPSA5KSwNCiAgICBheGlzLnRpdGxlLnkgPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oaGp1c3QgPSAwLjUsIGZhY2UgPSAiYm9sZCIsIHNpemUgPSA5KSwNCiAgICBheGlzLnRleHQueCA9IGdndGV4dDo6ZWxlbWVudF9tYXJrZG93bihoanVzdCA9IDAuNSwgc2l6ZSA9IDcsIGZhY2UgPSAnYm9sZCcpLA0KICAgIGF4aXMudGV4dC55ID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGhqdXN0ID0gMSwgc2l6ZSA9IDcsIGZhY2UgPSAnYm9sZCcpLA0KICAgIHN0cmlwLnRleHQgPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oZmFjZSA9ICJib2xkIiwgc2l6ZSA9IDgpLA0KICAgIGxlZ2VuZC5wb3NpdGlvbiA9ICJub25lIiwNCiAgICBwYW5lbC5ncmlkID0gZ2dwbG90Mjo6ZWxlbWVudF9ibGFuaygpLA0KICAgIGF4aXMubGluZSA9IGdncGxvdDI6OmVsZW1lbnRfbGluZShjb2xvciA9ICJibGFjayIpLA0KICAgIGF4aXMudGlja3MgPSBnZ3Bsb3QyOjplbGVtZW50X2xpbmUoY29sb3IgPSAnYmxhY2snKQ0KICApIC0+IG9yaWdpbmFsX3Bsb3QNCg0KDQoNCiMgKiBBZGRpbmcgYSB0YWJsZSBpbiB0aGUgZW1wdHkgc3BhY2UgYXQgdGhlIGJvdHRvbSByaWdodCAtLS0tLS0tLS0tLS0tLS0tLQ0KDQojIEN1c3RvbSB0aGVtZQ0KYm9va3RhYnNfdGhlbWUgPC0gdHRoZW1lX21pbmltYWwoDQogIGNvcmUgPSBsaXN0KA0KICAgIGZnX3BhcmFtcyA9IGxpc3QoZm9udGZhY2UgPSAicGxhaW4iLCBjZXggPSAwLjYsIGhqdXN0ID0gMCwgeCA9IDAuMDUpDQogICksDQogIGNvbGhlYWQgPSBsaXN0KA0KICAgIGZnX3BhcmFtcyA9IGxpc3QoZm9udGZhY2UgPSAiYm9sZCIsIGNleCA9IDAuNjUsIGhqdXN0ID0gMCwgeCA9IDAuMDUpLA0KICAgIGJnX3BhcmFtcyA9IGxpc3QoZmlsbCA9IE5BKSAgIyBObyBzaGFkaW5nLCBsaWtlIGJvb2t0YWJzDQogICksDQogIHBhZGRpbmcgPSB1bml0KGMoMS43LCAxLjcpLCAibW0iKSAgIyBUaWdodGVyIHBhZGRpbmcNCikNCg0KIyBCYXNlIHRhYmxlDQp0YWJsZV9ncm9iIDwtIGdyaWRFeHRyYTo6dGFibGVHcm9iKGludHJhX3dpZGUsIHJvd3MgPSBOVUxMLCB0aGVtZSA9IGJvb2t0YWJzX3RoZW1lKQ0KdGFibGVfZ3JvYiRoZWlnaHRzIDwtIGdncGxvdDI6OnVuaXQocmVwKDEsIG5yb3codGFibGVfZ3JvYikpLCAibGluZXMiKSAqIDAuNQ0KDQojIEFkZCB0aXRsZSByb3cNCnRpdGxlX2dyb2IgPC0gZ3JpZDo6dGV4dEdyb2IoDQogICJJbnRyYS1MTE0gQ29lZmZpY2llbnQgRXN0aW1hdGVzIiwNCiAgZ3AgPSBncmlkOjpncGFyKGZvbnRmYWNlID0gImJvbGQiLCBmb250c2l6ZSA9IDkpLA0KICB4ID0gMC41LCBoanVzdCA9IDAuNQ0KKQ0KDQojIEFkZCB0aXRsZSByb3cgYWJvdmUgdGhlIHRhYmxlDQp0YWJsZV9ncm9iIDwtIGd0YWJsZTo6Z3RhYmxlX2FkZF9yb3dzKHRhYmxlX2dyb2IsIGhlaWdodHMgPSB1bml0KDEuNSwgImxpbmVzIiksIHBvcyA9IDApDQp0YWJsZV9ncm9iIDwtIGd0YWJsZTo6Z3RhYmxlX2FkZF9ncm9iKA0KICB0YWJsZV9ncm9iLA0KICBncm9icyA9IHRpdGxlX2dyb2IsDQogIHQgPSAxLCBsID0gMSwgciA9IG5jb2wodGFibGVfZ3JvYikNCikNCg0KdGFibGVfd2lkdGggPC0gc3VtKHRhYmxlX2dyb2Ikd2lkdGhzKQ0KdGFibGVfaGVpZ2h0IDwtIHN1bSh0YWJsZV9ncm9iJGhlaWdodHMpDQoNCnBhZGRpbmcgPC0gZ2dwbG90Mjo6dW5pdCguNSwgIm1tIikNCg0KIyBDb2xvciBjb2RlIHRoZSB0YWJsZSANCmNvcmVfbGxtX2luZGljZXMgPC0gd2hpY2goDQogIHRhYmxlX2dyb2IkbGF5b3V0JG5hbWUgPT0gImNvcmUtZmciICYgDQogICAgdGFibGVfZ3JvYiRsYXlvdXQkbCA9PSAxICAjIGNvbHVtbiAxDQopDQoNCm1vZGVsX2NvbG9ycyA8LSByZXAoYygiIzgwODA4MCIsICIjQzMxNDJEIiksIGVhY2g9NykgICMgbWF0Y2hlcyBpbnRyYV93aWRlDQoNCmZvciAoaSBpbiBzZXFfYWxvbmcoY29yZV9sbG1faW5kaWNlcykpIHsNCiAgZ3JvYl9pbmRleCA8LSBjb3JlX2xsbV9pbmRpY2VzW2ldDQogIG9yaWdpbmFsX2dwIDwtIHRhYmxlX2dyb2IkZ3JvYnNbW2dyb2JfaW5kZXhdXSRncA0KICB0YWJsZV9ncm9iJGdyb2JzW1tncm9iX2luZGV4XV0kZ3AgPC0gbW9kaWZ5TGlzdChvcmlnaW5hbF9ncCwgZ3Bhcihjb2wgPSBtb2RlbF9jb2xvcnNbaV0pKQ0KfQ0KDQojIEFkZCBib3JkZXIgYXJvdW5kIGVudGlyZSB0YWJsZSB1c2luZyBncm9iVHJlZQ0KYm9yZGVyZWRfdGFibGUgPC0gZ3JpZDo6Z3JvYlRyZWUoDQogIGdyaWQ6OnJlY3RHcm9iKA0KICAgIHdpZHRoID0gdGFibGVfd2lkdGgrIHBhZGRpbmcsDQogICAgaGVpZ2h0ID0gdGFibGVfaGVpZ2h0ICsgcGFkZGluZywNCiAgICBncCA9IGdyaWQ6OmdwYXIoZmlsbCA9IE5BLCBsd2QgPSAwLjcsIGNvbCA9J2JsYWNrJykgDQogICksDQogIHRhYmxlX2dyb2INCikNCg0KDQpmaW5hbF9wbG90IDwtIGNvd3Bsb3Q6OmdnZHJhdygpICsNCiAgY293cGxvdDo6ZHJhd19wbG90KG9yaWdpbmFsX3Bsb3QsIDAsIDAsIDEsIDEpICsgICMgbWFpbiBwbG90IHRha2VzIGZ1bGwgYXJlYQ0KICBjb3dwbG90OjpkcmF3X2dyb2IoYm9yZGVyZWRfdGFibGUsIHggPSAwLjc0MiwgeSA9IDAuMTYsIHdpZHRoID0gMC4wMSwgaGVpZ2h0ID0gMC4wMSkNCg0KZmluYWxfcGxvdA0KYGBgDQoNCg0KIyMjIEludGVyLVJhdGVyIFJlbGlhYmlsaXR5IGFtb25nIExMTXMNCg0KQWZ0ZXIgY29uZmlybWluZyBzdHJvbmcgaW50ZXJuYWwgY29uc2lzdGVuY3kgd2l0aGluIGluZGl2aWR1YWwgbW9kZWxzLCB3ZSBleGFtaW5lIGFncmVlbWVudCBiZXR3ZWVuIGRpZmZlcmVudCBMTE1zLiBJbnN0ZWFkIG9mIGFuYWx5emluZyBhbGwgcG9zc2libGUgY29tYmluYXRpb25zIHdpdGhpbiBlYWNoIGdyb3VwLCB3ZSBmb2N1cyBvbiB0b3AtTiBjb21iaW5hdGlvbnMgcmFua2VkIGJ5IEtyaXBwZW5kb3JmZidzIEFscGhhLCB3aXRoIE4gcmFuZ2luZyBmcm9tIDIgdG8gNy4gICANCg0KYGBge3IgaW50ZXJfbGxtLCBjbGFzcy5zb3VyY2UgPSAnZm9sZC1oaWRlJywgbWVzc2FnZT1GQUxTRSwgZmlnLmFsaWduPSdjZW50ZXInLCBmaWcud2lkdGg9NiwgZmlnLmhlaWdodD04LCBmaWcuZnVsbHdpZHRoPVRSVUV9DQoNCiMgLS0tLSBTZXR1cCAtLS0tDQpjaGF0X21vZGVscyA9IGMoDQogICJUb3AgMiBDaGVhcGVyIiwNCiAgIlRvcCAyIEV4cGVuc2l2ZSIsDQogICJUb3AgMyBDaGVhcGVyIiwNCiAgIlRvcCAzIEV4cGVuc2l2ZSIsDQogICJUb3AgNCBDaGVhcGVyIiwNCiAgIlRvcCA0IEV4cGVuc2l2ZSIsDQogICJUb3AgNSBDaGVhcGVyIiwNCiAgIlRvcCA1IEV4cGVuc2l2ZSIsDQogICJUb3AgNiBDaGVhcGVyIiwNCiAgIlRvcCA2IEV4cGVuc2l2ZSIsDQogICJUb3AgNyBDaGVhcGVyIiwNCiAgIlRvcCA3IEV4cGVuc2l2ZSINCikNCg0KbW9kZWxfYWJicmV2ID0gYygNCiAgIlRvcCAyIENoZWFwZXIiICAgPSAiVG9wIDIgQ2hlYXBlciIsDQogICJUb3AgMiBFeHBlbnNpdmUiID0gIlRvcCAyIEV4cGVuc2l2ZSIsDQogICJUb3AgMyBDaGVhcGVyIiAgID0gIlRvcCAzIENoZWFwZXIiLA0KICAiVG9wIDMgRXhwZW5zaXZlIiA9ICJUb3AgMyBFeHBlbnNpdmUiLA0KICAiVG9wIDQgQ2hlYXBlciIgICA9ICJUb3AgNCBDaGVhcGVyIiwNCiAgIlRvcCA0IEV4cGVuc2l2ZSIgPSAiVG9wIDQgRXhwZW5zaXZlIiwNCiAgIlRvcCA1IENoZWFwZXIiICAgPSAiVG9wIDUgQ2hlYXBlciIsDQogICJUb3AgNSBFeHBlbnNpdmUiID0gIlRvcCA1IEV4cGVuc2l2ZSIsDQogICJUb3AgNiBDaGVhcGVyIiAgID0gIlRvcCA2IENoZWFwZXIiLA0KICAiVG9wIDYgRXhwZW5zaXZlIiA9ICJUb3AgNiBFeHBlbnNpdmUiLA0KICAiVG9wIDcgQ2hlYXBlciIgICA9ICJUb3AgNyBDaGVhcGVyIiwNCiAgIlRvcCA3IEV4cGVuc2l2ZSIgPSAiVG9wIDcgRXhwZW5zaXZlIg0KKQ0KDQpjaGF0X21vZGVsX2FiYnJldiA9IGMoDQogICJwaGk0LW1pbmkiICAgICAgICAgICAgICAgICA9ICJwaGk0LW1pbmkiLA0KICAicGhpNDpsYXRlc3QiICAgICAgICAgICAgICA9ICJwaGk0OmxhdGVzdCIsDQogICJsbGFtYTMuMjoxQiIgICAgICAgICAgICAgID0gImxsYW1hMy4yOjFCIiwNCiAgImxsYW1hMy4yOjNCIiAgICAgICAgICAgICAgPSAibGxhbWEzLjI6M0IiLA0KICAiZ2VtbWEzOjFCIiAgICAgICAgICAgICAgICA9ICJnZW1tYTM6MUIiLA0KICAiZ2VtbWEzOjI3QiIgICAgICAgICAgICAgICA9ICJnZW1tYTM6MjdCIiwNCiAgImRlZXBzZWVrLXIxOjEuNUIiICAgICAgICAgPSAiZGVlcHNlZWstcjE6MS41QiIsDQogICJkZWVwc2Vlay1yMTo3QiIgICAgICAgICAgID0gImRlZXBzZWVrLXIxOjdCIiwNCiAgImNvbW1hbmQtcjdiIiAgICAgICAgICAgICAgPSAiY29tbWFuZC1yN2IiLA0KICAiY29tbWFuZC1yLXBsdXMtMDgtMjAyNCIgICA9ICJjb21tYW5kLXItcGx1cyIsDQogICJncHQtNG8tbWluaS0yMDI0LTA3LTE4IiAgID0gImdwdC00by1taW5pIiwNCiAgImdwdC00by0yMDI0LTExLTIwIiAgICAgICAgPSAiZ3B0LTRvIiwNCiAgImNsYXVkZS0zLTUtaGFpa3UtMjAyNDEwMjIiPSAiY2xhdWRlLTMtNS1oYWlrdSIsDQogICJjbGF1ZGUtMy03LXNvbm5ldC0yMDI1MDIxOSIgPSAiY2xhdWRlLTMtNy1zb25uZXQiDQopDQoNCmNoYXRfbW9kZWxfY29sb3JzID0gcmVwKGMoJyM4MDgwODAnLCAnI0MzMTQyRCcpLCBsZW5ndGgub3V0ID0gbGVuZ3RoKGNoYXRfbW9kZWxzKSkNCmNoYXRfbW9kZWxfY29sb3JzID0gc3RhdHM6OnNldE5hbWVzKGNoYXRfbW9kZWxfY29sb3JzLCBtb2RlbF9hYmJyZXZbY2hhdF9tb2RlbHNdKQ0KDQoNCm1vZGVsX2xhYmVscyA8LSBwdXJycjo6bWFwX2Nocihtb2RlbF9hYmJyZXZbY2hhdF9tb2RlbHNdLCB+ew0KICBpZiAoZ3JlcGwoIl5zcGFjZXIiLCAueCkpIHJldHVybigiIikNCiAgZ2x1ZTo6Z2x1ZSgiPHNwYW4gc3R5bGU9J2NvbG9yOntjaGF0X21vZGVsX2NvbG9yc1sueF19Jz48Yj57Lnh9PC9iPjwvc3Bhbj4iKQ0KfSkNCg0KbmFtZXMobW9kZWxfbGFiZWxzKSA8LSBtb2RlbF9hYmJyZXZbY2hhdF9tb2RlbHNdDQoNCg0KIyAtLS0tIExvYWQgRGF0YSAtLS0tDQppbnRyYV9kZiA9IHJlYWRfY3N2KCIuLi9yZXN1bHRzL2JpbmFyeV9yZWxpYWJpbGl0eV9tZXRyaWNzLmNzdiIpICU+JSANCiAgZmlsdGVyKGNvZWZmLm5hbWUgPT0gIktyaXBwZW5kb3JmZidzIEFscGhhIikgJT4lIA0KICBzZWxlY3QobW9kZWwsIGNvZWZmLnZhbCwgY29lZmYuc2UpDQoNCmludHJhX2RmJG1vZGVsIDwtIGZhY3RvcihpbnRyYV9kZiRtb2RlbCwgbGV2ZWxzID0gYygicGhpNC1taW5pIiwgICAgICAgICAgICAgICAgIA0KICAicGhpNDpsYXRlc3QiLCAibGxhbWEzLjI6MUIiLCAibGxhbWEzLjI6M0IiLCAiZ2VtbWEzOjFCIiwgICAgICAgICAgICAgICAgDQogICJnZW1tYTM6MjdCIiwgImRlZXBzZWVrLXIxOjEuNUIiLCAiZGVlcHNlZWstcjE6N0IiLCAgICAgICAgICAgDQogICJjb21tYW5kLXI3YiIsICJjb21tYW5kLXItcGx1cy0wOC0yMDI0IiwNCiAgImdwdC00by1taW5pLTIwMjQtMDctMTgiLCAiZ3B0LTRvLTIwMjQtMTEtMjAiLCAgICAgICAgDQogICJjbGF1ZGUtMy01LWhhaWt1LTIwMjQxMDIyIiwgImNsYXVkZS0zLTctc29ubmV0LTIwMjUwMjE5IikNCiAgKQ0KDQppbnRyYV9kZiA8LSBpbnRyYV9kZiAlPiUgDQogIGFycmFuZ2UobW9kZWwpICU+JSANCiAgbXV0YXRlKGNvc3QgPSByZXAoYygiQ2hlYXBlciIsICJFeHBlbnNpdmUiKSwgNykpICU+JSANCiAgc2VsZWN0KGNvc3QsIGV2ZXJ5dGhpbmcoKSkgJT4lIA0KICBhcnJhbmdlKGNvc3QsIGRlc2MoY29lZmYudmFsKSkNCg0KYmluYXJ5X2FuYWx5c2lzX2RmIDwtIHJlYWRfY3N2KCIuLi9yZXN1bHRzL2JpbmFyeV9hbmFseXNpc19kZi5jc3YiKQ0KDQpkZl9sb25nIDwtIGJpbmFyeV9hbmFseXNpc19kZiAlPiUNCiAgcGl2b3RfbG9uZ2VyKGNvbHMgPSBzdGFydHNfd2l0aCgicmVwIiksIG5hbWVzX3RvID0gInJlcCIsIHZhbHVlc190byA9ICJ2YWx1ZSIpDQoNCmludGVyX3RlbXAgPC0gZGZfbG9uZyAlPiUNCiAgc2VsZWN0KGNoYXRfbW9kZWwsIHZhbHVlKSAlPiUNCiAgZ3JvdXBfYnkoY2hhdF9tb2RlbCkgJT4lDQogIG11dGF0ZShpZCA9IHJvd19udW1iZXIoKSkgJT4lDQogIHBpdm90X3dpZGVyKG5hbWVzX2Zyb20gPSBjaGF0X21vZGVsLCB2YWx1ZXNfZnJvbSA9IHZhbHVlKSAlPiUgDQogIHNlbGVjdCgtaWQpDQoNCmZpbmRfaW50ZXJfbWV0cmljcyA8LSBmdW5jdGlvbihrLCBjb3N0KXsNCiAgbW9kZWxzIDwtIGFzLmNoYXJhY3RlcihpbnRyYV9kZiRtb2RlbFtpbnRyYV9kZiRjb3N0PT1jb3N0XSlbMTprXQ0KICB0ZW1wIDwtIGludGVyX3RlbXBbbW9kZWxzXQ0KICB0ZW1wJGNoYXRfbW9kZWwgPSBwYXN0ZSgiVG9wIiwgaywgY29zdCkNCiAgcmVsaWFiaWxpdHlfY29lZnModGVtcCwgMTprKSAlPiUgDQogICAgbXV0YXRlKGxvd2VyX2NpID0gY29lZmYudmFsIC0gc3RhdHM6OnFub3JtKDAuOTc1KSAqIGNvZWZmLnNlLA0KICAgICAgICAgICB1cHBlcl9jaSA9IGNvZWZmLnZhbCArIHN0YXRzOjpxbm9ybSgwLjk3NSkgKiBjb2VmZi5zZSkgJT4lIA0KICAgIHNlbGVjdChtb2RlbCwgY29lZmYubmFtZSwgY29lZmYudmFsLCBsb3dlcl9jaSwgdXBwZXJfY2kpDQp9DQoNCiMgRGVmaW5lIHZhbHVlcyB0byBsb29wIG92ZXINCmtfdmFscyA8LSAyOjcNCmNvc3RfbGV2ZWxzIDwtIGMoIkNoZWFwZXIiLCAiRXhwZW5zaXZlIikNCg0KDQojIENyZWF0ZSBhbGwgY29tYmluYXRpb25zIG9mIGsgYW5kIGNvc3QNCmNvbWJpbmF0aW9ucyA8LSBleHBhbmQuZ3JpZChrID0gMjo3LCBjb3N0ID0gYygiQ2hlYXBlciIsICJFeHBlbnNpdmUiKSwgS0VFUC5PVVQuQVRUUlMgPSBGQUxTRSkNCg0KIyBBcHBseSB0aGUgZnVuY3Rpb24gZm9yIGVhY2ggY29tYmluYXRpb24NCmludGVyX2RmIDwtIHBtYXBfZGZyKGNvbWJpbmF0aW9ucywgZnVuY3Rpb24oaywgY29zdCkgew0KICBmaW5kX2ludGVyX21ldHJpY3MoaywgY29zdCkgJT4lDQogICAgbXV0YXRlKGsgPSBrLCBjb3N0ID0gY29zdCkNCn0pDQoNCmludGVyX2RmJGFiYnJldiA8LSBmYWN0b3IobW9kZWxfYWJicmV2W2ludGVyX2RmJG1vZGVsXSkNCg0KaW50cmFfd2lkZSA8LSBpbnRyYV9kZiAlPiUgDQogIG11dGF0ZShSYW5rID0gcmVwKHBhc3RlKCJUb3AiLCAxOjcpLCAyKSwNCiAgICAgICAgIExMTSA9IGNoYXRfbW9kZWxfYWJicmV2W21vZGVsXQ0KICApICU+JSANCiAgc2VsZWN0KFJhbmssIExMTSkgDQoNCmludGVyX3dpZGUgPC0gZGF0YS5mcmFtZShSYW5rID0gMTo3LA0KICAgICAgICAgICAgICAgICAgICAgICAgIENoZWFwZXIgPSBpbnRyYV93aWRlJExMTVsxOjddLA0KICAgICAgICAgICAgICAgICAgICAgICAgIEV4cGVuc2l2ZSA9IGludHJhX3dpZGUkTExNWzg6MTRdKSANCg0KDQpyb3cubmFtZXMoaW50ZXJfd2lkZSkgPSBOVUxMDQoNCg0KIyAqUGxvdCB0aGUgRGF0YSAtLS0tLS0tTlVMTCMgKlBsb3QgdGhlIERhdGEgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KDQppbnRlcl9kZiB8PiANCiAgZ2dwbG90Mjo6Z2dwbG90KGdncGxvdDI6OmFlcyh4ID0gYWJicmV2LCB5ID0gY29lZmYudmFsKSkgKw0KICBnZ3Bsb3QyOjpnZW9tX2JsYW5rKCkgKw0KICBnZ3Bsb3QyOjpnZW9tX3BvaW50KGdncGxvdDI6OmFlcyhjb2xvciA9IGFiYnJldiksIHNpemUgPSAxLjUsIG5hLnJtID0gVFJVRSkgKw0KICBnZ3Bsb3QyOjpnZW9tX2Vycm9yYmFyKA0KICAgIGdncGxvdDI6OmFlcyh5bWluID0gbG93ZXJfY2ksIHltYXggPSB1cHBlcl9jaSwgY29sb3IgPSBhYmJyZXYpLA0KICAgIHdpZHRoID0gMC41LA0KICAgIG5hLnJtID0gVFJVRQ0KICApICsNCiAgIyBBbm5vdGF0ZSBsb3dlciBDSSB2YWx1ZXMNCiAgZ2dwbG90Mjo6Z2VvbV90ZXh0KA0KICAgIGRhdGEgPSBpbnRlcl9kZiwNCiAgICBnZ3Bsb3QyOjphZXMoDQogICAgICB5ID0gbG93ZXJfY2ksDQogICAgICBsYWJlbCA9IHNwcmludGYoIiUuM2YiLCBsb3dlcl9jaSksDQogICAgICBjb2xvciA9IGFiYnJldg0KICAgICksDQogICAgc2l6ZSA9IDIuMjUsDQogICAgaGp1c3QgPSAxLjIsDQogICAgbmEucm0gPSBUUlVFDQogICkgKw0KICANCiAgIyBBbm5vdGF0ZSB1cHBlciBDSSB2YWx1ZXMNCiAgZ2dwbG90Mjo6Z2VvbV90ZXh0KA0KICAgIGRhdGEgPSBpbnRlcl9kZiwNCiAgICBnZ3Bsb3QyOjphZXMoDQogICAgICB5ID0gdXBwZXJfY2ksDQogICAgICBsYWJlbCA9IHNwcmludGYoIiUuM2YiLCB1cHBlcl9jaSksDQogICAgICBjb2xvciA9IGFiYnJldg0KICAgICksDQogICAgc2l6ZSA9IDIuMjUsDQogICAgaGp1c3QgPSAtMC4yLA0KICAgIG5hLnJtID0gVFJVRQ0KICApICsNCiAgZ2dwbG90Mjo6c2NhbGVfY29sb3JfbWFudWFsKHZhbHVlcyA9IGNoYXRfbW9kZWxfY29sb3JzLCBuYS50cmFuc2xhdGUgPSBGQUxTRSkgKw0KICBnZ3Bsb3QyOjpzY2FsZV94X2Rpc2NyZXRlKGxhYmVscyA9IG1vZGVsX2xhYmVscykgKw0KICBnZ3Bsb3QyOjpzY2FsZV95X2NvbnRpbnVvdXMobGltaXRzID0gYygwLjYxNSwgMC45NTMpLCBicmVha3MgPSBzZXEoMC42NSwgMC45LCAwLjA1KSkgKw0KICBnZ3Bsb3QyOjpjb29yZF9mbGlwKCkgKw0KICBnZ3Bsb3QyOjpmYWNldF93cmFwKH4gY29lZmYubmFtZSwgbmNvbCA9IDIsIHNjYWxlcyA9ICJmaXhlZCIsIGF4ZXMgPSAnYWxsJykgKw0KICBnZ3Bsb3QyOjpsYWJzKA0KICAgIHRpdGxlID0gIkludGVyLUxMTSBSZWxpYWJpbGl0eSIsDQogICAgc3VidGl0bGUgPSAiPHNwYW4gc3R5bGU9J2NvbG9yOiM4MDgwODAnPkNoZWFwZXI8L3NwYW4+IHZzLiA8c3BhbiBzdHlsZT0nY29sb3I6I0MzMTQyRCc+bW9yZSBleHBlbnNpdmUgKHRpbWUsIGNvc3QpIDwvc3Bhbj4gTExNcyBieSBjb21wYW55IiwNCiAgICB4ID0gTlVMTCwNCiAgICB5ID0gTlVMTA0KICApICsNCiAgZ2dwbG90Mjo6dGhlbWVfbWluaW1hbCgpICsNCiAgZ2dwbG90Mjo6dGhlbWUoDQogICAgcGxvdC50aXRsZSA9IGdndGV4dDo6ZWxlbWVudF9tYXJrZG93bihoanVzdCA9IDAuNSwgZmFjZSA9ICJib2xkIiwgc2l6ZSA9IDkpLA0KICAgIHBsb3Quc3VidGl0bGUgPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oaGp1c3QgPSAwLjUsIGZhY2UgPSAiYm9sZCIsIHNpemUgPSA5KSwNCiAgICBheGlzLnRpdGxlLnggPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oaGp1c3QgPSAwLjUsIGZhY2UgPSAiYm9sZCIsIHNpemUgPSA5KSwNCiAgICBheGlzLnRpdGxlLnkgPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oaGp1c3QgPSAwLjUsIGZhY2UgPSAiYm9sZCIsIHNpemUgPSA5KSwNCiAgICBheGlzLnRleHQueCA9IGdndGV4dDo6ZWxlbWVudF9tYXJrZG93bihoanVzdCA9IDAuNSwgc2l6ZSA9IDcsIGZhY2UgPSAnYm9sZCcpLA0KICAgIGF4aXMudGV4dC55ID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGhqdXN0ID0gMSwgc2l6ZSA9IDcsIGZhY2UgPSAnYm9sZCcpLA0KICAgIHN0cmlwLnRleHQgPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oZmFjZSA9ICJib2xkIiwgc2l6ZSA9IDgpLA0KICAgIGxlZ2VuZC5wb3NpdGlvbiA9ICJub25lIiwNCiAgICBwYW5lbC5ncmlkID0gZ2dwbG90Mjo6ZWxlbWVudF9ibGFuaygpLA0KICAgIGF4aXMubGluZSA9IGdncGxvdDI6OmVsZW1lbnRfbGluZShjb2xvciA9ICJibGFjayIpLA0KICAgIGF4aXMudGlja3MgPSBnZ3Bsb3QyOjplbGVtZW50X2xpbmUoY29sb3IgPSAnYmxhY2snKQ0KICApIC0+IG9yaWdpbmFsX3Bsb3QNCg0KIyAqIEFkZGluZyBhIHRhYmxlIGluIHRoZSBlbXB0eSBzcGFjZSBhdCB0aGUgYm90dG9tIHJpZ2h0IC0tLS0tLS0tLS0tLS0tLS0tDQoNCiMgQ3VzdG9tIHRoZW1lDQpib29rdGFic190aGVtZSA8LSB0dGhlbWVfbWluaW1hbCgNCiAgY29yZSA9IGxpc3QoDQogICAgZmdfcGFyYW1zID0gbGlzdChmb250ZmFjZSA9ICJwbGFpbiIsIGNleCA9IDAuNiwgaGp1c3QgPSAwLCB4ID0gMC4wNSkNCiAgKSwNCiAgY29saGVhZCA9IGxpc3QoDQogICAgZmdfcGFyYW1zID0gbGlzdChmb250ZmFjZSA9ICJib2xkIiwgY2V4ID0gMC42NSwgaGp1c3QgPSAwLCB4ID0gMC4wNSksDQogICAgYmdfcGFyYW1zID0gbGlzdChmaWxsID0gTkEpICAjIE5vIHNoYWRpbmcsIGxpa2UgYm9va3RhYnMNCiAgKSwNCiAgcGFkZGluZyA9IHVuaXQoYyg0LjUsIDQuNSksICJtbSIpICAjIFRpZ2h0ZXIgcGFkZGluZw0KKQ0KDQojIEJhc2UgdGFibGUNCnRhYmxlX2dyb2IgPC0gZ3JpZEV4dHJhOjp0YWJsZUdyb2IoaW50ZXJfd2lkZSwgcm93cyA9IE5VTEwsIHRoZW1lID0gYm9va3RhYnNfdGhlbWUpDQp0YWJsZV9ncm9iJGhlaWdodHMgPC0gZ2dwbG90Mjo6dW5pdChyZXAoMSwgbnJvdyh0YWJsZV9ncm9iKSksICJsaW5lcyIpKjAuNw0KDQoNCiMgQWRkIHRpdGxlIHJvdw0KdGl0bGVfZ3JvYiA8LSBncmlkOjp0ZXh0R3JvYigNCiAgIkxMTSBSYW5rIGJhc2VkIG9uIEtyaXBlbmRvcmZmJ3MgQWxwaGEiLA0KICBncCA9IGdyaWQ6OmdwYXIoZm9udGZhY2UgPSAiYm9sZCIsIGZvbnRzaXplID0gOSksDQogIHggPSAwLjUsIGhqdXN0ID0gMC41DQopDQoNCiMgQWRkIHRpdGxlIHJvdyBhYm92ZSB0aGUgdGFibGUNCnRhYmxlX2dyb2IgPC0gZ3RhYmxlOjpndGFibGVfYWRkX3Jvd3ModGFibGVfZ3JvYiwgaGVpZ2h0cyA9IHVuaXQoMS41LCAibGluZXMiKSwgcG9zID0gMCkNCnRhYmxlX2dyb2IgPC0gZ3RhYmxlOjpndGFibGVfYWRkX2dyb2IoDQogIHRhYmxlX2dyb2IsDQogIGdyb2JzID0gdGl0bGVfZ3JvYiwNCiAgdCA9IDEsIGwgPSAxLCByID0gbmNvbCh0YWJsZV9ncm9iKQ0KKQ0KDQp0YWJsZV93aWR0aCA8LSBzdW0odGFibGVfZ3JvYiR3aWR0aHMpDQp0YWJsZV9oZWlnaHQgPC0gc3VtKHRhYmxlX2dyb2IkaGVpZ2h0cykNCg0KcGFkZGluZyA8LSBnZ3Bsb3QyOjp1bml0KC41LCAibW0iKQ0KDQojIENvbG9yIGNvZGUgdGhlIHRhYmxlIA0KY29yZV9sbG1faW5kaWNlcyA8LSB3aGljaCgNCiAgdGFibGVfZ3JvYiRsYXlvdXQkbmFtZSA9PSAiY29yZS1mZyIgJiANCiAgICB0YWJsZV9ncm9iJGxheW91dCRsID09IDEgICMgY29sdW1uIDENCikNCg0KIyBGaW5kIGFsbCBjb3JlIChib2R5IGNlbGwpIGdyb2JzDQpjb3JlX2luZGljZXMgPC0gd2hpY2godGFibGVfZ3JvYiRsYXlvdXQkbmFtZSA9PSAiY29yZS1mZyIpDQoNCiMgTG9vcCBvdmVyIGVhY2ggY29yZSBjZWxsIGFuZCBjb2xvciBieSBjb2x1bW4gYW5kIGNlbnRlciB0aGUgZmlyc3QgY29sdW1uDQpmb3IgKGkgaW4gY29yZV9pbmRpY2VzKSB7DQogIGNvbF9udW0gPC0gdGFibGVfZ3JvYiRsYXlvdXRbaSwgImwiXQ0KICBvcmlnaW5hbF9ncCA8LSB0YWJsZV9ncm9iJGdyb2JzW1tpXV0kZ3ANCiAgZ3JvYiA8LSB0YWJsZV9ncm9iJGdyb2JzW1tpXV0gICMgY3VycmVudCBjZWxsIGdyb2INCiAgDQogIGlmIChjb2xfbnVtID09IDEpIHsNCiAgICAjIENlbnRlci1hbGlnbiBjb2x1bW4gMSAoUmFuaykNCiAgICBncm9iJGhqdXN0IDwtIDAuNQ0KICAgIGdyb2IkeCA8LSB1bml0KDAuNSwgIm5wYyIpDQogIH0gZWxzZSBpZiAoY29sX251bSA9PSAyKSB7DQogICAgIyBDaGVhcGVyIExMTSBjb2x1bW4g4oCUIGdyYXkNCiAgICBncm9iJGdwIDwtIG1vZGlmeUxpc3Qob3JpZ2luYWxfZ3AsIGdwYXIoY29sID0gIiM4MDgwODAiKSkNCiAgfSBlbHNlIGlmIChjb2xfbnVtID09IDMpIHsNCiAgICAjIEV4cGVuc2l2ZSBMTE0gY29sdW1uIOKAlCByZWQNCiAgICBncm9iJGdwIDwtIG1vZGlmeUxpc3Qob3JpZ2luYWxfZ3AsIGdwYXIoY29sID0gIiNDMzE0MkQiKSkNCiAgfQ0KICANCiAgdGFibGVfZ3JvYiRncm9ic1tbaV1dIDwtIGdyb2INCn0NCg0KDQojIEFkZCBib3JkZXIgYXJvdW5kIGVudGlyZSB0YWJsZSB1c2luZyBncm9iVHJlZQ0KYm9yZGVyZWRfdGFibGUgPC0gZ3JpZDo6Z3JvYlRyZWUoDQogIGdyaWQ6OnJlY3RHcm9iKA0KICAgIHdpZHRoID0gdGFibGVfd2lkdGggKyBwYWRkaW5nLA0KICAgIGhlaWdodCA9IHRhYmxlX2hlaWdodCArIHBhZGRpbmcsDQogICAgZ3AgPSBncmlkOjpncGFyKGZpbGwgPSBOQSwgbHdkID0gMC43LCBjb2wgPSdibGFjaycpIA0KICApLA0KICB0YWJsZV9ncm9iDQopDQoNCg0KZmluYWxfcGxvdCA8LSBjb3dwbG90OjpnZ2RyYXcoKSArDQogIGNvd3Bsb3Q6OmRyYXdfcGxvdChvcmlnaW5hbF9wbG90LCAwLCAwLCAxLCAxKSArICAjIG1haW4gcGxvdCB0YWtlcyBmdWxsIGFyZWENCiAgY293cGxvdDo6ZHJhd19ncm9iKGJvcmRlcmVkX3RhYmxlLCB4ID0gMC43NDIsIHkgPSAwLjE2LCB3aWR0aCA9IDAuMDEsIGhlaWdodCA9IDAuMDEpDQoNCmZpbmFsX3Bsb3QNCmBgYA0KDQojIyBWYWxpZGl0eSBvZiB0aGUgQmluYXJ5IENsYXNzaWZpY2F0aW9uIExhYmVscyAoQ29tcGFyaXNvbiB3aXRoICJTdG9ja05ld3NBUEkiIExhYmVscykNCg0KSW4gdGhpcyBzZWN0aW9uLCB3ZSB3aWxsIGNvbXBhcmUgdGhlIExMTSBsYWJlbHMgd2l0aCB0aGUgbGFiZWxzIHByb3ZpZGVkIGJ5IHRoZSBTdG9ja05ld3NBUEkgKCoqYmVuY2htYXJrKiopLiBXZSB3aWxsIGNhbGN1bGF0ZSB0aGUgYWdyZWVtZW50IGJldHdlZW4gdGhlIExMTSBsYWJlbHMgYW5kIHRoZSAiYmVuY2htYXJrIiBsYWJlbHMgdXNpbmcgdmFyaW91cyByZWxpYWJpbGl0eSBtZXRyaWNzLiBXZSB3aWxsIGFsc28gYXNzZXNzIHRoZSB2YWxpZGl0eSBvZiB0aGUgTExNIGxhYmVscyBieSBjb21wYXJpbmcgdGhlbSB3aXRoIHRoZSAiYmVuY2htYXJrIiBsYWJlbHMuDQoNCmBgYHtyIGJpbmFyeV9zZW5zX2NsYXNzaWZpY2F0aW9uX3ZhbGlkaXR5LCByZXN1bHRzPSdob2xkJywgbWVzc2FnZT1GQUxTRX0NCnN0b2NrX3NlbnRpbWVudF9kZiA9IHJlYWRyOjpyZWFkX2NzdigiLi4vZGF0YS9iaW5hcnlfY2xhc3NpZmljYXRpb25fZGF0YS5jc3YiKSB8PiANCiAgZHBseXI6OnNlbGVjdChkYXRlLCB0aXRsZSwgdGV4dCwgdGlja2Vycywgc2VudGltZW50KQ0KDQp2YWxpZGl0eV9kZiA9IA0KICByZWFkcjo6cmVhZF9jc3YoIi4uL3Jlc3VsdHMvYmluYXJ5X2FuYWx5c2lzX2RmLmNzdiIpIHw+IA0KICBkcGx5cjo6c2VsZWN0KC1wZXJjZW50X2FncmVlbWVudCkgfD4gDQogIGRwbHlyOjpsZWZ0X2pvaW4oc3RvY2tfc2VudGltZW50X2RmLCBieSA9IGMoImRhdGUiLCAidGl0bGUiLCAidGV4dCIsICJ0aWNrZXJzIikpIHw+IA0KICAjIHBlcmNlbnQgb2YgcmVwXzEsIHJlcF8yLCAuLiwgYWdyZWVtZW50IHdpdGggdGhlIGJlbmNobWFyaw0KICBkcGx5cjo6cm93d2lzZSgpIHw+DQogIGRwbHlyOjptdXRhdGUoDQogICAgYWdyZWVtZW50X25hX3BlbmFsdHkgPSBjYWxjdWxhdGVfYWdyZWVtZW50KA0KICAgICAgcmVwcyA9IGRwbHlyOjpjX2Fjcm9zcyhkcGx5cjo6c3RhcnRzX3dpdGgoInJlcF8iKSksDQogICAgICBncm91bmRfdHJ1dGggPSBkcGx5cjo6Y19hY3Jvc3MoZHBseXI6OmNvbnRhaW5zKCJzZW50aW1lbnQiKSkNCiAgICApLA0KICAgIGFncmVlbWVudF9uYV9hc19uYSA9IGNhbGN1bGF0ZV9hZ3JlZW1lbnQoDQogICAgICByZXBzID0gZHBseXI6OmNfYWNyb3NzKGRwbHlyOjpzdGFydHNfd2l0aCgicmVwXyIpKSwNCiAgICAgIGdyb3VuZF90cnV0aCA9IGRwbHlyOjpjX2Fjcm9zcyhkcGx5cjo6Y29udGFpbnMoInNlbnRpbWVudCIpKSwNCiAgICAgIG5hID0gTkENCiAgICApDQogICkgIA0KDQojIHNhdmUgdGhlIHZhbGlkaXR5IGRhdGFmcmFtZSBhcyBSRFMgYW5kIENTViBmaWxlcw0KcmVhZHI6OndyaXRlX3Jkcyh2YWxpZGl0eV9kZiwgIi4uL3Jlc3VsdHMvYmluYXJ5X3NlbnNfdmFsaWRpdHlfZGYucmRzIikNCnJlYWRyOjp3cml0ZV9jc3YodmFsaWRpdHlfZGYsICIuLi9yZXN1bHRzL2JpbmFyeV9zZW5zX3ZhbGlkaXR5X2RmLmNzdiIpDQpgYGANCg0KIyMjIFZpc3VhbGl6aW5nIHRoZSBWYWxpZGl0eSBNZXRyaWNzDQoNCldlIHJlcG9ydCBhY2N1cmFjeSwgdHJ1ZSBwb3NpdGl2ZSByYXRlLCB0cnVlIG5lZ2F0aXZlIHJhdGUsIHBvc2l0aXZlIHByZWRpY3RpdmUgdmFsdWUsIGFuZCBGMSBzY29yZSB0byBwcm92aWRlIGEgY29tcHJlaGVuc2l2ZSBwZXJmb3JtYW5jZS4gVGhlIGZpZ3VyZSBiZWxvdyBzaG93cyB0aGUgTExNIGNsYXNzaWZpY2F0aW9uIHBlcmZvcm1hbmNlIHYucy4gYmVuY2htYXJrICh0aG9zZSBvYnRhaW5lZCBmcm9tIHRoZSBgU3RvY2tOZXdzQVBJYCkuIFRoZSBkb3QgcmVwcmVzZW50cyBlYWNoIG1ldHJpYydzIG1lYW4gdmFsdWUgKGZyb20gaXRzIGludHJhLXJhdGVyIGZpdmUgcmVwbGljYXRlcyksIGFuZCB0aGUgd2hpc2tlcnMgbGVuZ3RoIHJlZmxlY3QgdGhlIHN0YW5kYXJkIGVycm9yLiAgIA0KDQpgYGB7ciB2YWxpZGl0eV9TdG9ja05ld3NBUEksIGNsYXNzLnNvdXJjZSA9ICdmb2xkLWhpZGUnLCBtZXNzYWdlPUZBTFNFLCBmaWcuYWxpZ249J2NlbnRlcicsIGZpZy53aWR0aD02LCBmaWcuaGVpZ2h0PTgsIGZpZy5mdWxsd2lkdGg9VFJVRX0NCiMgLS0tLSBTZXR1cCAtLS0tDQpjaGF0X21vZGVscyA9IGMoDQogICJwaGk0LW1pbmkiLA0KICAicGhpNDpsYXRlc3QiLA0KICAibGxhbWEzLjI6MUIiLA0KICAibGxhbWEzLjI6M0IiLCANCiAgImdlbW1hMzoxQiIsDQogICJnZW1tYTM6MjdCIiwNCiAgImRlZXBzZWVrLXIxOjEuNUIiLA0KICAiZGVlcHNlZWstcjE6N0IiLA0KICAiY29tbWFuZC1yN2IiLA0KICAiY29tbWFuZC1yLXBsdXMtMDgtMjAyNCIsDQogICJncHQtNG8tbWluaS0yMDI0LTA3LTE4IiwNCiAgImdwdC00by0yMDI0LTExLTIwIiwNCiAgImNsYXVkZS0zLTUtaGFpa3UtMjAyNDEwMjIiLA0KICAiY2xhdWRlLTMtNy1zb25uZXQtMjAyNTAyMTkiDQopDQoNCm1vZGVsX2FiYnJldiA9IGMoDQogICJwaGk0LW1pbmkiICAgICAgICAgICAgICAgICA9ICJwaGk0LW1pbmkiLA0KICAicGhpNDpsYXRlc3QiICAgICAgICAgICAgICA9ICJwaGk0OmxhdGVzdCIsDQogICJsbGFtYTMuMjoxQiIgICAgICAgICAgICAgID0gImxsYW1hMy4yOjFCIiwNCiAgImxsYW1hMy4yOjNCIiAgICAgICAgICAgICAgPSAibGxhbWEzLjI6M0IiLA0KICAiZ2VtbWEzOjFCIiAgICAgICAgICAgICAgICA9ICJnZW1tYTM6MUIiLA0KICAiZ2VtbWEzOjI3QiIgICAgICAgICAgICAgICA9ICJnZW1tYTM6MjdCIiwNCiAgImRlZXBzZWVrLXIxOjEuNUIiICAgICAgICAgPSAiZGVlcHNlZWstcjE6MS41QiIsDQogICJkZWVwc2Vlay1yMTo3QiIgICAgICAgICAgID0gImRlZXBzZWVrLXIxOjdCIiwNCiAgImNvbW1hbmQtcjdiIiAgICAgICAgICAgICAgPSAiY29tbWFuZC1yN2IiLA0KICAiY29tbWFuZC1yLXBsdXMtMDgtMjAyNCIgICA9ICJjb21tYW5kLXItcGx1cyIsDQogICJncHQtNG8tbWluaS0yMDI0LTA3LTE4IiAgID0gImdwdC00by1taW5pIiwNCiAgImdwdC00by0yMDI0LTExLTIwIiAgICAgICAgPSAiZ3B0LTRvIiwNCiAgImNsYXVkZS0zLTUtaGFpa3UtMjAyNDEwMjIiPSAiY2xhdWRlLTMtNS1oYWlrdSIsDQogICJjbGF1ZGUtMy03LXNvbm5ldC0yMDI1MDIxOSIgPSAiY2xhdWRlLTMtNy1zb25uZXQiDQopDQoNCmNoYXRfbW9kZWxfY29sb3JzID0gcmVwKGMoJyM4MDgwODAnLCAnI0MzMTQyRCcpLCBsZW5ndGgub3V0ID0gbGVuZ3RoKGNoYXRfbW9kZWxzKSkNCmNoYXRfbW9kZWxfY29sb3JzID0gc3RhdHM6OnNldE5hbWVzKGNoYXRfbW9kZWxfY29sb3JzLCBtb2RlbF9hYmJyZXZbY2hhdF9tb2RlbHNdKQ0KDQoNCm1vZGVsX2xhYmVscyA8LSBwdXJycjo6bWFwX2NociggbW9kZWxfYWJicmV2W2NoYXRfbW9kZWxzXSwgfnsNCiAgaWYgKGdyZXBsKCJec3BhY2VyIiwgLngpKSByZXR1cm4oIiIpDQogIGdsdWU6OmdsdWUoIjxzcGFuIHN0eWxlPSdjb2xvcjp7Y2hhdF9tb2RlbF9jb2xvcnNbLnhdfSc+PGI+ey54fTwvYj48L3NwYW4+IikNCn0pDQpuYW1lcyhtb2RlbF9sYWJlbHMpIDwtIG1vZGVsX2FiYnJldltjaGF0X21vZGVsc10NCg0KDQoNCg0KIyAtLS0tIExvYWQgRGF0YSAtLS0tDQppbnRlcl9kZiA9IA0KICB0aWJibGU6OnRyaWJibGUoDQogICAgfk1vZGVsLCAgICAgICAgICAgICB+QWNjdXJhY3ksIH5BY2NfU0UsIH5UUFIsICAgIH5UUFJfU0UsIH5UTlIsICAgIH5UTlJfU0UsIH5QUFYsICAgIH5QUFZfU0UsIH5GMSwgICAgfkYxX1NFLA0KICAgICJwaGk0LW1pbmkiLCAgICAgICAgMC44MzAsICAgICAwLjAwMSwgICAwLjgxNywgICAwLjAwMSwgICAwLjg0MywgICAwLjAwMywgICAwLjgzOSwgICAwLjAwMywgIDAuODI4LCAgIDAuMDAxLA0KICAgICJwaGk0OmxhdGVzdCIsICAgICAgMC44MjIsICAgICAwLjAwMSwgICAwLjg0MiwgICAwLjAwMywgICAwLjgwMiwgICAwLjAwMSwgICAwLjgwOSwgICAwLjAwMCwgIDAuODI1LCAgIDAuMDAxLA0KICAgICJsbGFtYTMuMjoxQiIsICAgICAgMC44NjQsICAgICAwLjAwNSwgICAwLjg4MSwgICAwLjAwMywgICAwLjg0NywgICAwLjAwNiwgICAwLjg1MiwgICAwLjAwNSwgIDAuODY3LCAgIDAuMDA0LA0KICAgICJsbGFtYTMuMjozQiIsICAgICAgMC43NjMsICAgICAwLjAwMywgICAwLjc5NywgICAwLjAwMCwgICAwLjcyOCwgICAwLjAwNSwgICAwLjc0NiwgICAwLjAwNCwgIDAuNzcwLCAgIDAuMDAyLA0KICAgICJnZW1tYTM6MUIiLCAgICAgICAgMC44NzYsICAgICAwLjAwMiwgICAwLjgxOCwgICAwLjAwOCwgICAwLjkzMywgICAwLjAwNSwgICAwLjkyNCwgICAwLjAwNCwgIDAuODY4LCAgIDAuMDAzLA0KICAgICJnZW1tYTM6MjdCIiwgICAgICAgMC44MjMsICAgICAwLjAwMSwgICAwLjgxOSwgICAwLjAwMywgICAwLjgyOCwgICAwLjAwMSwgICAwLjgyNiwgICAwLjAwMSwgIDAuODIzLCAgIDAuMDAxLA0KICAgICJkZWVwc2Vlay1yMToxLjVCIiwgMC43NzUsICAgICAwLjAwOCwgICAwLjcyNSwgICAwLjAxNSwgICAwLjgyNSwgICAwLjAwMiwgICAwLjgwNiwgICAwLjAwNSwgIDAuNzY0LCAgIDAuMDEwLA0KICAgICJkZWVwc2Vlay1yMTo3QiIsICAgMC44MzIsICAgICAwLjAwMiwgICAwLjgzNSwgICAwLjAwNSwgICAwLjgyOSwgICAwLjAwMSwgICAwLjgzMCwgICAwLjAwMSwgIDAuODMyLCAgIDAuMDAzLA0KICAgICJjb21tYW5kLXItcGx1cyIsICAgMC43NzQsICAgICAwLjAwMiwgICAwLjcxNSwgICAwLjAwMiwgICAwLjgzMiwgICAwLjAwNiwgICAwLjgxMCwgICAwLjAwNSwgIDAuNzYwLCAgIDAuMDAyLA0KICAgICJjb21tYW5kLXI3YiIsICAgICAgMC44NDcsICAgICAwLjAwMywgICAwLjg4MywgICAwLjAwNCwgICAwLjgxMCwgICAwLjAwMywgICAwLjgyMywgICAwLjAwMywgIDAuODUyLCAgIDAuMDAzLA0KICAgICJncHQtNG8tbWluaSIsICAgICAgMC44MjAsICAgICAwLjAwMywgICAwLjgwMCwgICAwLjAwNSwgICAwLjg0MCwgICAwLjAwMiwgICAwLjgzNCwgICAwLjAwMywgIDAuODE3LCAgIDAuMDA0LA0KICAgICJncHQtNG8iLCAgICAgICAgICAgMC43ODcsICAgICAwLjAwNywgICAwLjc0MywgICAwLjAxMSwgICAwLjgzMiwgICAwLjAwMywgICAwLjgxNSwgICAwLjAwNCwgIDAuNzc3LCAgIDAuMDA4LA0KICAgICJjbGF1ZGUtMy01LWhhaWt1IiwgMC44NDgsICAgICAwLjAwMiwgICAwLjg4MiwgICAwLjAwNCwgICAwLjgxNSwgICAwLjAwMywgICAwLjgyNiwgICAwLjAwMiwgIDAuODUzLCAgIDAuMDAyLA0KICAgICJjbGF1ZGUtMy03LXNvbm5ldCIsMC44MzEsICAgICAwLjAwMywgICAwLjg2NywgICAwLjAwNCwgICAwLjc5NSwgICAwLjAwMywgICAwLjgwOSwgICAwLjAwMiwgIDAuODM3LCAgIDAuMDAzDQogICkNCg0KaW50ZXJfZGYgPSBpbnRlcl9kZiB8PiANCiAgZHBseXI6OnJlbmFtZShBY2N1cmFjeV9tZWFuID0gQWNjdXJhY3ksDQogICAgICAgICAgICAgICAgVFBSX21lYW4gPSBUUFIsDQogICAgICAgICAgICAgICAgVE5SX21lYW4gPSBUTlIsDQogICAgICAgICAgICAgICAgUFBWX21lYW4gPSBQUFYsDQogICAgICAgICAgICAgICAgRjFfbWVhbiA9IEYxKSB8Pg0KICBkcGx5cjo6cmVuYW1lKEFjY3VyYWN5X3NlID0gQWNjX1NFLA0KICAgICAgICAgICAgICAgIFRQUl9zZSA9IFRQUl9TRSwNCiAgICAgICAgICAgICAgICBUTlJfc2UgPSBUTlJfU0UsDQogICAgICAgICAgICAgICAgUFBWX3NlID0gUFBWX1NFLA0KICAgICAgICAgICAgICAgIEYxX3NlID0gRjFfU0UpDQoNCg0KaW50ZXJfbG9uZyA9IGludGVyX2RmIHw+IA0KICB0aWR5cjo6cGl2b3RfbG9uZ2VyKA0KICAgIGNvbHMgPSAtTW9kZWwsDQogICAgbmFtZXNfdG8gPSBjKCJNZXRyaWMiLCAiLnZhbHVlIiksDQogICAgbmFtZXNfc2VwID0gIl8iDQogICkgfD4NCiAgZHBseXI6Om11dGF0ZSgNCiAgICBsb3dlciA9IG1lYW4gLSBzZS9zcXJ0KDUpLA0KICAgIHVwcGVyID0gbWVhbiArIHNlL3NxcnQoNSksDQogICAgTW9kZWwgPSBmYWN0b3IoTW9kZWwsIGxldmVscyA9IHJldihtb2RlbF9hYmJyZXZbY2hhdF9tb2RlbHNdKSkNCiAgICApDQoNCnRhYmxlX2RmIDwtIGludGVyX2RmIHw+DQogIGRwbHlyOjpzZWxlY3QoTW9kZWwsIEFjY3VyYWN5X21lYW4sIFRQUl9tZWFuLCBUTlJfbWVhbiwgUFBWX21lYW4sIEYxX21lYW4pICU+JQ0KICBkcGx5cjo6cmVuYW1lKA0KICAgIExMTSA9IE1vZGVsLA0KICAgIEFDQyA9IEFjY3VyYWN5X21lYW4sDQogICAgVFBSID0gVFBSX21lYW4sDQogICAgVE5SID0gVE5SX21lYW4sDQogICAgUFBWID0gUFBWX21lYW4sDQogICAgRjEgPSBGMV9tZWFuKSB8Pg0KICBkcGx5cjo6bXV0YXRlKGRwbHlyOjphY3Jvc3MoZHBseXI6OndoZXJlKGlzLm51bWVyaWMpLCB+IHNwcmludGYoIiUuMmYiLCAuKSkNCiAgKQ0KDQoNCiMgKlBsb3QgdGhlIERhdGEgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KDQppbnRlcl9sb25nIHw+IA0KICBnZ3Bsb3QyOjpnZ3Bsb3QoZ2dwbG90Mjo6YWVzKHggPSBNb2RlbCwgeSA9IG1lYW4sIGNvbG9yID0gTW9kZWwpKSArDQogIGdncGxvdDI6Omdlb21fYmxhbmsoKSArDQogIGdncGxvdDI6Omdlb21fcG9pbnQoc2l6ZSA9IDAuOSwgbmEucm0gPSBUUlVFKSArDQogIGdncGxvdDI6Omdlb21fZXJyb3JiYXIoDQogICAgZ2dwbG90Mjo6YWVzKHltaW4gPSBsb3dlciwgeW1heCA9IHVwcGVyKSwNCiAgICB3aWR0aCA9IDAuNCwNCiAgICBuYS5ybSA9IFRSVUUNCiAgKSArDQogIGdncGxvdDI6Omdlb21fdGV4dChnZ3Bsb3QyOjphZXMobGFiZWwgPSBzcHJpbnRmKCIlLjNmIiwgbG93ZXIpLCB5ID0gbG93ZXIpLCBzaXplID0gMi4yNSwgaGp1c3QgPSAxLjIsIG5hLnJtID0gVFJVRSkgKw0KICBnZ3Bsb3QyOjpnZW9tX3RleHQoZ2dwbG90Mjo6YWVzKGxhYmVsID0gc3ByaW50ZigiJS4zZiIsIHVwcGVyKSwgeSA9IHVwcGVyKSwgc2l6ZSA9IDIuMjUsIGhqdXN0ID0gLTAuMiwgbmEucm0gPSBUUlVFKSArDQogIGdncGxvdDI6OnNjYWxlX2NvbG9yX21hbnVhbCh2YWx1ZXMgPSBjaGF0X21vZGVsX2NvbG9ycywgbmEudHJhbnNsYXRlID0gRkFMU0UpICsNCiAgZ2dwbG90Mjo6c2NhbGVfeF9kaXNjcmV0ZShsYWJlbHMgPSBtb2RlbF9sYWJlbHMpICsNCiAgZ2dwbG90Mjo6c2NhbGVfeV9jb250aW51b3VzKGxpbWl0cyA9IGMoMC42NSwgMS4wMiksIGJyZWFrcyA9IHNlcSgwLjcsIDEuMCwgMC4xKSkgKw0KICBnZ3Bsb3QyOjpjb29yZF9mbGlwKCkgKw0KICBnZ3Bsb3QyOjpmYWNldF93cmFwKH4gTWV0cmljLCBuY29sID0gMiwgc2NhbGVzID0gImZpeGVkIiwgYXhlcyA9ICdhbGwnKSArDQogIGdncGxvdDI6OmxhYnMoDQogICAgdGl0bGUgPSAiQ2xhc3NpZmljYXRpb24gUGVyZm9ybWFuY2UgdnMuIFN0b2NrTmV3c0FQSSBMYWJlbHMiLA0KICAgIHN1YnRpdGxlID0gIjxzcGFuIHN0eWxlPSdjb2xvcjojODA4MDgwJz5DaGVhcGVyPC9zcGFuPiB2cy4gPHNwYW4gc3R5bGU9J2NvbG9yOiNDMzE0MkQnPm1vcmUgZXhwZW5zaXZlICh0aW1lLCBjb3N0KSA8L3NwYW4+IExMTXMgYnkgY29tcGFueSIsDQogICAgeCA9IE5VTEwsDQogICAgeSA9IE5VTEwNCiAgKSArDQogIGdncGxvdDI6OnRoZW1lX21pbmltYWwoKSArDQogIGdncGxvdDI6OnRoZW1lKA0KICAgIHBsb3QudGl0bGUgPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oaGp1c3QgPSAwLjUsIGZhY2UgPSAiYm9sZCIsIHNpemUgPSA5KSwNCiAgICBwbG90LnN1YnRpdGxlID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGhqdXN0ID0gMC41LCBmYWNlID0gImJvbGQiLCBzaXplID0gOCksDQogICAgYXhpcy50aXRsZS54ID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGhqdXN0ID0gMC41LCBmYWNlID0gImJvbGQiLCBzaXplID0gOCksDQogICAgYXhpcy50aXRsZS55ID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGhqdXN0ID0gMC41LCBmYWNlID0gImJvbGQiLCBzaXplID0gOCksDQogICAgYXhpcy50ZXh0LnggPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oaGp1c3QgPSAwLjUsIHNpemUgPSA3LCBmYWNlID0gJ2JvbGQnKSwNCiAgICBheGlzLnRleHQueSA9IGdndGV4dDo6ZWxlbWVudF9tYXJrZG93bihoanVzdCA9IDEsIHNpemUgPSA3LCBmYWNlID0gJ2JvbGQnKSwNCiAgICBzdHJpcC50ZXh0ID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGZhY2UgPSAiYm9sZCIsIHNpemUgPSA4KSwNCiAgICBsZWdlbmQucG9zaXRpb24gPSAibm9uZSIsDQogICAgcGFuZWwuZ3JpZCA9IGdncGxvdDI6OmVsZW1lbnRfYmxhbmsoKSwNCiAgICBheGlzLmxpbmUgPSBnZ3Bsb3QyOjplbGVtZW50X2xpbmUoY29sb3IgPSAiYmxhY2siKSwNCiAgICBheGlzLnRpY2tzID0gZ2dwbG90Mjo6ZWxlbWVudF9saW5lKGNvbG9yID0gJ2JsYWNrJykNCiAgKSAtPiBvcmlnaW5hbF9wbG90DQoNCg0KDQojICogQWRkaW5nIGEgdGFibGUgaW4gdGhlIGVtcHR5IHNwYWNlIGF0IHRoZSBib3R0b20gcmlnaHQgLS0tLS0tLS0tLS0tLS0tLS0NCg0KIyBDdXN0b20gdGhlbWUNCmJvb2t0YWJzX3RoZW1lIDwtIHR0aGVtZV9taW5pbWFsKA0KICBjb3JlID0gbGlzdCgNCiAgICBmZ19wYXJhbXMgPSBsaXN0KGZvbnRmYWNlID0gInBsYWluIiwgY2V4ID0gMC41NSwgaGp1c3QgPSAwLCB4ID0gMC4wNSkNCiAgKSwNCiAgY29saGVhZCA9IGxpc3QoDQogICAgZmdfcGFyYW1zID0gbGlzdChmb250ZmFjZSA9ICJib2xkIiwgY2V4ID0gMC42LCBoanVzdCA9IDAsIHggPSAwLjA1KSwNCiAgICBiZ19wYXJhbXMgPSBsaXN0KGZpbGwgPSBOQSkgICMgTm8gc2hhZGluZywgbGlrZSBib29rdGFicw0KICApLA0KICBwYWRkaW5nID0gdW5pdChjKDEuNywgMS43KSwgIm1tIikgICMgVGlnaHRlciBwYWRkaW5nDQopDQoNCiMgQmFzZSB0YWJsZQ0KdGFibGVfZ3JvYiA8LSBncmlkRXh0cmE6OnRhYmxlR3JvYih0YWJsZV9kZiwgcm93cyA9IE5VTEwsIHRoZW1lID0gYm9va3RhYnNfdGhlbWUpDQp0YWJsZV9ncm9iJGhlaWdodHMgPC0gZ2dwbG90Mjo6dW5pdChyZXAoMSwgbnJvdyh0YWJsZV9ncm9iKSksICJsaW5lcyIpICogMC41DQoNCiMgQWRkIHRpdGxlIHJvdw0KdGl0bGVfZ3JvYiA8LSBncmlkOjp0ZXh0R3JvYigNCiAgIlZhbGlkaXR5IHdpdGggQmVuY2htYXJrIE1vZGVsIiwNCiAgZ3AgPSBncmlkOjpncGFyKGZvbnRmYWNlID0gImJvbGQiLCBmb250c2l6ZSA9IDkpLA0KICB4ID0gMC41LCBoanVzdCA9IDAuNQ0KKQ0KDQojIEFkZCB0aXRsZSByb3cgYWJvdmUgdGhlIHRhYmxlDQp0YWJsZV9ncm9iIDwtIGd0YWJsZTo6Z3RhYmxlX2FkZF9yb3dzKHRhYmxlX2dyb2IsIGhlaWdodHMgPSB1bml0KDEuNSwgImxpbmVzIiksIHBvcyA9IDApDQp0YWJsZV9ncm9iIDwtIGd0YWJsZTo6Z3RhYmxlX2FkZF9ncm9iKA0KICB0YWJsZV9ncm9iLA0KICBncm9icyA9IHRpdGxlX2dyb2IsDQogIHQgPSAxLCBsID0gMSwgciA9IG5jb2wodGFibGVfZ3JvYikNCikNCg0KdGFibGVfd2lkdGggPC0gc3VtKHRhYmxlX2dyb2Ikd2lkdGhzKQ0KdGFibGVfaGVpZ2h0IDwtIHN1bSh0YWJsZV9ncm9iJGhlaWdodHMpDQoNCnBhZGRpbmcgPC0gZ2dwbG90Mjo6dW5pdCguNSwgIm1tIikNCg0KIyBDb2xvciBjb2RlIHRoZSB0YWJsZSANCmNvcmVfbGxtX2luZGljZXMgPC0gd2hpY2goDQogIHRhYmxlX2dyb2IkbGF5b3V0JG5hbWUgPT0gImNvcmUtZmciICYgDQogICAgdGFibGVfZ3JvYiRsYXlvdXQkbCA9PSAxICAjIGNvbHVtbiAxDQopDQoNCm1vZGVsX2NvbG9ycyA8LSByZXAoYygiIzgwODA4MCIsICIjQzMxNDJEIiksIGxlbmd0aC5vdXQgPSAxNCkgICMgbWF0Y2hlcyBpbnRlcl93aWRlDQoNCmZvciAoaSBpbiBzZXFfYWxvbmcoY29yZV9sbG1faW5kaWNlcykpIHsNCiAgZ3JvYl9pbmRleCA8LSBjb3JlX2xsbV9pbmRpY2VzW2ldDQogIG9yaWdpbmFsX2dwIDwtIHRhYmxlX2dyb2IkZ3JvYnNbW2dyb2JfaW5kZXhdXSRncA0KICB0YWJsZV9ncm9iJGdyb2JzW1tncm9iX2luZGV4XV0kZ3AgPC0gbW9kaWZ5TGlzdChvcmlnaW5hbF9ncCwgZ3Bhcihjb2wgPSBtb2RlbF9jb2xvcnNbaV0pKQ0KfQ0KDQojIEFkZCBib3JkZXIgYXJvdW5kIGVudGlyZSB0YWJsZSB1c2luZyBncm9iVHJlZQ0KYm9yZGVyZWRfdGFibGUgPC0gZ3JpZDo6Z3JvYlRyZWUoDQogIGdyaWQ6OnJlY3RHcm9iKA0KICAgIHdpZHRoID0gdGFibGVfd2lkdGgrIHBhZGRpbmcsDQogICAgaGVpZ2h0ID0gdGFibGVfaGVpZ2h0ICsgcGFkZGluZywNCiAgICBncCA9IGdyaWQ6OmdwYXIoZmlsbCA9IE5BLCBsd2QgPSAwLjcsIGNvbCA9J2JsYWNrJykgDQogICksDQogIHRhYmxlX2dyb2INCikNCg0KDQpmaW5hbF9wbG90IDwtIGNvd3Bsb3Q6OmdnZHJhdygpICsNCiAgY293cGxvdDo6ZHJhd19wbG90KG9yaWdpbmFsX3Bsb3QsIDAsIDAsIDEsIDEpICsgICMgbWFpbiBwbG90IHRha2VzIGZ1bGwgYXJlYQ0KICBjb3dwbG90OjpkcmF3X2dyb2IoYm9yZGVyZWRfdGFibGUsIHggPSAwLjc0MiwgeSA9IDAuMTYsIHdpZHRoID0gMC4wMSwgaGVpZ2h0ID0gMC4wMSkNCg0KZmluYWxfcGxvdA0KDQpgYGANCg0KIyMgVmFsaWRpdHkgb2YgdGhlIEJpbmFyeSBDbGFzc2lmaWNhdGlvbiBMYWJlbHMgKENvbXBhcmlzb24gd2l0aCAiRXh0ZXJuYWwgQ3JpdGVyaW9uIiBJbXBhY3RzKSB7I3ZhbGlkaXR5X2Jpbl9nb2xkfQ0KDQpJbiB0aGlzIHNlY3Rpb24sIHdlIHdpbGwgcmVwZWF0IG91ciBhbmFseXNpcyBmcm9tIHRoZSBwcmV2aW91cyBzZWN0aW9uLCBidXQgdGhpcyB0aW1lIHdlIHdpbGwgY29tcGFyZSB0aGUgTExNIGxhYmVscyB3aXRoIHRoZSAiRXh0ZXJuYWwgQ3JpdGVyaW9uIiBlZmZlY3RzIHRvIHdoYXQgYWN0dWFsbHkgaGFwcGVuZWQgdG8gdGhlIHN0b2NrIHByaWNlcy4gV2Ugd2lsbCBjYWxjdWxhdGUgdGhlIGFncmVlbWVudCBiZXR3ZWVuIHRoZSBMTE0gbGFiZWxzIGFuZCB0aGUgIkV4dGVybmFsIENyaXRlcmlvbiIgbGFiZWxzIHVzaW5nIG91ciBhZ3JlZW1lbnQgbWV0cmljcy4NCg0KDQojIyMgRXh0cmFjdGluZyB0aGUgRXh0ZXJuYWwgQ3JpdGVyaW9uIFN0b2NrIFBlcmZvcm1hbmNlDQoNCkluIHRoZSBjb2RlIGNodW5rIGJlbG93LCB3ZSB1c2UgUiBhbmQgdGhlIGB0aWR5cXVhbnRgIHBhY2thZ2UgdG8gZXh0cmFjdCB0aGUgc3RvY2sgcHJpY2VzIGZvciB0aGUgdGlja2VycyBhc3NvY2lhdGVkIHdpdGggdGhlIGJpbmFyeSBjbGFzc2lmaWNhdGlvbiBsYWJlbHMuIFdlIHRoZW4gc2F2ZSB0aGUgcmF3IHN0b2NrIHByaWNlcyB0byBhIENTViBmaWxlIGZvciBmdXJ0aGVyIGFuYWx5c2lzLg0KDQpgYGB7ciBiaW5hcnlfZ3JvdW5kX3RydXRoX3N0b2NrX3BlcmZvcm1hbmNlLCBldmFsPUZBTFNFLCBjYWNoZSA9IFRSVUUsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0UsIHJlc3VsdHM9J2hpZGUnfQ0KdGlja2Vyc19kZiA9IHJlYWRyOjpyZWFkX2NzdigNCiAgZmlsZSA9ICIuLi9kYXRhL2JpbmFyeV9jbGFzc2lmaWNhdGlvbl9kYXRhLmNzdiINCikNCg0KdGlja2Vyc19saXN0ID0gdGlja2Vyc19kZiR0aWNrZXJzIHw+IHVuaXF1ZSgpDQptaW5fZGF0ZSA9IG1pbih0aWNrZXJzX2RmJGRhdGUpDQptYXhfZGF0ZSA9IG1heCh0aWNrZXJzX2RmJGRhdGUpDQoNCg0Kc3RvY2tfcHJpY2VzID0gdGlkeXF1YW50Ojp0cV9nZXQoDQogIHggPSB0aWNrZXJzX2xpc3QsDQogIGZyb20gPSBtaW5fZGF0ZSwNCiAgdG8gPSBtYXhfZGF0ZSwNCiAgZ2V0ID0gInN0b2NrLnByaWNlcyINCikNCg0Kc3RvY2tfcHJpY2VzJHN5bWJvbCB8PiB1bmlxdWUoKSB8PiBsZW5ndGgoKQ0KDQpvdGhlcl9zdG9ja3MgPSBjKCJNR0MiLCAiXkdTUEMiKQ0KDQojQUhULCBDUldWLCBNVFRSIChhZnRlciAyMDI1LTAzLTAzIG5vdCBhdmFpbGFibGUpDQoNCm90aGVyX3ByaWNlcyA9IHRpZHlxdWFudDo6dHFfZ2V0KA0KICB4ID0gb3RoZXJfc3RvY2tzLA0KICBmcm9tID0gbWluX2RhdGUsDQogIHRvID0gbWF4X2RhdGUsDQogIGdldCA9ICJzdG9jay5wcmljZXMiDQopIA0KDQpzdG9ja19wcmljZXMgfD4NCiAgZHBseXI6OnNlbGVjdChzeW1ib2wsIGRhdGUsIGFkanVzdGVkKSB8Pg0KICBkcGx5cjo6cmVuYW1lKHRpY2tlciA9IHN5bWJvbCkgfD4gDQogIGRwbHlyOjpiaW5kX3Jvd3Mob3RoZXJfcHJpY2VzKSB8Pg0KICBkcGx5cjo6YXJyYW5nZSh0aWNrZXIsIGRhdGUpIHw+IA0KICByZWFkcjo6d3JpdGVfY3N2KA0KICAgICIuLi9yZXN1bHRzL3N0b2NrX3ByaWNlc19yYXcuY3N2Ig0KICApDQoNCmBgYA0KDQojIyMgQ29tcHV0aW5nIHRoZSBMZWFkaW5nIE5ld3MgRWZmZWN0DQoNCmBgYHtyIGJpbmFyeV9sZWFkaW5nX25ld3NfZWZmZWN0LCByZXN1bHRzPSdob2xkJywgbWVzc2FnZT1GQUxTRX0NCiMgcmVhZCB0aGUgc3RvY2sgcHJpY2VzIGFuZCBjb21wdXRlIHRoZSByZXR1cm5zDQpyZXR1cm5zX2RmID0gDQogIHJlYWRyOjpyZWFkX2NzdigiLi4vcmVzdWx0cy9zdG9ja19wcmljZXNfcmF3LmNzdiIpIHw+IA0KICBkcGx5cjo6Z3JvdXBfYnkodGlja2VyKSB8Pg0KICBkcGx5cjo6bXV0YXRlKA0KICAgIHJldHVybiA9IChhZGp1c3RlZCAtIGRwbHlyOjpsYWcoYWRqdXN0ZWQpKSAvIGRwbHlyOjpsYWcoYWRqdXN0ZWQpLA0KICAgIHBlcmNfcmV0dXJuID0gcmV0dXJuICogMTAwDQogICkgfD4gDQogIGRwbHlyOjp1bmdyb3VwKCkNCg0KIyBjYWxjdWxhdGUgdGhlIGJhc2VsaW5lIHMmcCByZXR1cm5zDQpzcDUwMF9yZXR1cm5zID0gcmV0dXJuc19kZiB8Pg0KICBkcGx5cjo6ZmlsdGVyKHRpY2tlciA9PSAnXkdTUEMnKSB8Pg0KICBkcGx5cjo6c2VsZWN0KGRhdGUsIHNwNTAwX3BlcmNfcmV0dXJuID0gcGVyY19yZXR1cm4pDQoNCiMgY29tYmluZSB0aGF0IHdpdGggdGhlIHN0b2NrIHJldHVybnMNCnJldHVybnNfZGYgPSByZXR1cm5zX2RmIHw+IA0KICBkcGx5cjo6bGVmdF9qb2luKHNwNTAwX3JldHVybnMsIGJ5ID0gImRhdGUiKSB8PiANCiAgZHBseXI6OmZpbHRlcih0aWNrZXIgIT0gJ15HU1BDJykgfD4NCiAgZHBseXI6Om11dGF0ZSgNCiAgICBleGNlc3NfcmV0dXJuID0gcGVyY19yZXR1cm4gLSBzcDUwMF9wZXJjX3JldHVybiwNCiAgICBleGNlc3NfcmV0dXJuX3NpZ24gPSBkcGx5cjo6Y2FzZV93aGVuKA0KICAgICAgZXhjZXNzX3JldHVybiA+IDAgfiAiUG9zaXRpdmUiLA0KICAgICAgZXhjZXNzX3JldHVybiA8IDAgfiAiTmVnYXRpdmUiLA0KICAgICAgVFJVRSB+IE5BDQogICAgKSwNCiAgICBsZWFkaW5nX25ld3NfZWZmZWN0ID0gZHBseXI6OmxlYWQoZXhjZXNzX3JldHVybl9zaWduKQ0KICApDQoNCiMgc2F2ZSB0aGUgcmV0dXJucyBkYXRhZnJhbWUgYXMgUkRTIGFuZCBDU1YgZmlsZXMNCnJlYWRyOjp3cml0ZV9yZHMocmV0dXJuc19kZiwgIi4uL3Jlc3VsdHMvc3RvY2tfcmV0dXJuc19kZi5yZHMiKQ0KcmVhZHI6OndyaXRlX2NzdihyZXR1cm5zX2RmLCAiLi4vcmVzdWx0cy9zdG9ja19yZXR1cm5zX2RmLmNzdiIpDQoNCiMgc2hvdyB0aGUgcmVzdWx0cyBmb3IgdGhlIGZpcnN0IDUgdGlja2VycyBvZiBpbnRlcmVzdCAoZXhjbHVkaW5nIFMmUCA1MDApDQpEVDo6ZGF0YXRhYmxlKA0KICByZXR1cm5zX2RmIHw+IA0KICAgIGRwbHlyOjpmaWx0ZXIoIHRpY2tlciAlaW4lIHVuaXF1ZShyZXR1cm5zX2RmJHRpY2tlcilbMTo1XSApLCANCiAgcm93bmFtZXMgPSBGQUxTRSwgDQogIGV4dGVuc2lvbnMgPSBjKCJGaXhlZENvbHVtbnMiKSwNCiAgb3B0aW9ucyA9IGxpc3QoDQogICAgcGFnZUxlbmd0aCA9IDEwLCANCiAgICBzY3JvbGxYID0gVFJVRSwNCiAgICBmaXhlZENvbHVtbnMgPSBsaXN0KGxlZnRDb2x1bW5zID0gMSkNCiAgKQ0KKSB8PiANCiAgRFQ6OmZvcm1hdFJvdW5kKGNvbHVtbnMgPSAzOjcsIGRpZ2l0cyA9IDIpIHw+IA0KICBEVDo6Zm9ybWF0U3R5bGUoDQogICAgY29sdW1ucyA9ICdsZWFkaW5nX25ld3NfZWZmZWN0JywgICMgQXBwbHkgdG8gdGhlIDl0aCBjb2x1bW4gKG5ld3NfZWZmZWN0KQ0KICAgIHZhbHVlQ29sdW1ucyA9ICdsZWFkaW5nX25ld3NfZWZmZWN0JywNCiAgICBjb2xvciA9IERUOjpKUygidmFsdWUgPT0gJ1Bvc2l0aXZlJyA/ICcjMkM3QkI2JyA6ICcjRDcxOTFDICciKSwgDQogICAgZm9udFdlaWdodCA9ICdib2xkJyAgICAgICAjIEJvbGQgdGhlIHRleHQNCiAgKQ0KYGBgDQoNCg0KIyMjIE1lcmdpbmcgdGhlIGBOZXdzIEVmZmVjdGAgd2l0aCBPdXIgQmluYXJ5IENsYXNzaWZpY2F0aW9uIExhYmVscw0KDQpJbiB0aGUgY29kZSBjaHVuayBiZWxvdywgd2UgbWVyZ2UgdGhlIGJpbmFyeSBjbGFzc2lmaWNhdGlvbiBsYWJlbHMgd2l0aCB0aGUgbGVhZGluZyBuZXdzIGVmZmVjdC4gV2UgYWxzbyBjYWxjdWxhdGUgdGhlIGRpc3RyaWJ1dGlvbiBvZiB0aGUgbGVhZGluZyBuZXdzIGVmZmVjdCBhbmQgaWRlbnRpZnkgdGhlIHRpY2tlcnMgd2l0aCBtaXNzaW5nIHZhbHVlcy4NCg0KYGBge3IgYmluYXJ5X21lcmdpbmdfbmV3c19lZmZlY3RzLCByZXN1bHRzPSdhc2lzJ30NCiMgcmVhZCB0aGUgc3RvY2sgcmV0dXJucyBkYXRhZnJhbWUNCmFkanVzdGVkX25ld3NfZGYgPSANCiAgcmVhZHI6OnJlYWRfY3N2KCIuLi9yZXN1bHRzL2JpbmFyeV9hbmFseXNpc19kZi5jc3YiKSB8PiANCiAgZHBseXI6Om11dGF0ZShkYXRlID0gbHVicmlkYXRlOjphc19kYXRlKGRhdGUpKSB8PiANCiAgZHBseXI6OnNlbGVjdCgtcGVyY2VudF9hZ3JlZW1lbnQpDQoNCiMgTllTRSBob2xpZGF5cyB3aXRoaW4gdGhlIGRhdGUgcmFuZ2UNCm1pbl9kYXRlID0gbWluKGFkanVzdGVkX25ld3NfZGYkZGF0ZSkgfD4gYXMuRGF0ZSgpDQptYXhfZGF0ZSA9IG1heChhZGp1c3RlZF9uZXdzX2RmJGRhdGUpIHw+IGFzLkRhdGUoKQ0KDQojIG55c2UgaG9saWRheXMgd2l0aGluIG91ciByYW5nZSBkYXRlcw0KbnlzZV9ob2xpZGF5cyA9IA0KICAgdGltZURhdGU6OmhvbGlkYXlOWVNFKHllYXIgPSBsdWJyaWRhdGU6OnllYXIobWluX2RhdGUpOmx1YnJpZGF0ZTo6eWVhcihtYXhfZGF0ZSkpIHw+IA0KICAgYXMuZGF0YS5mcmFtZSgpIHw+IA0KICAgZHBseXI6OnJlbmFtZShvZmZfZGF0ZXMgPSAxKSB8Pg0KICAgZHBseXI6Om11dGF0ZShvZmZfZGF0ZXMgPSBhcy5EYXRlKG9mZl9kYXRlcykpIHw+IA0KICAgZHBseXI6OmZpbHRlcihvZmZfZGF0ZXMgPj0gbWluX2RhdGUgJiBvZmZfZGF0ZXMgPD0gbWF4X2RhdGUpIHw+ICANCiAgIGRwbHlyOjpwdWxsKG9mZl9kYXRlcykgDQoNCiMgd2Vla2VuZHMgd2l0aGluIHRoZSBkYXRlIHJhbmdlDQp3ZWVrZW5kcyA9DQogIGRhdGEuZnJhbWUob2ZmX2RhdGVzID0gc2VxLkRhdGUobWluX2RhdGUsIG1heF9kYXRlLCBieSA9ICJkYXkiKSkgfD4NCiAgZHBseXI6Om11dGF0ZSgNCiAgICB3ZWVrZGF5ID0gbHVicmlkYXRlOjp3ZGF5KG9mZl9kYXRlcywgbGFiZWwgPSBUUlVFLCBhYmJyID0gRkFMU0UpDQogICAgKSB8PiANCiAgZHBseXI6OmZpbHRlcih3ZWVrZGF5ICVpbiUgYygnU2F0dXJkYXknLCAnU3VuZGF5JykgKSB8PiANCiAgZHBseXI6OnB1bGwob2ZmX2RhdGVzKQ0KDQojIGNvbWJpbmUgdGhlIGhvbGlkYXlzIGFuZCB3ZWVrZW5kcw0KIyBvZmZfZGF0ZXMgPSBjKG55c2VfaG9saWRheXMsIHdlZWtlbmRzKSB8PiBhcy5EYXRlKCkNCm9mZl9kYXRlcyA9IHdlZWtlbmRzDQoNCiMgdHJhZGluZyBkYXlzDQp0cmFkaW5nX2RheXMgPSBzZXEuRGF0ZShtaW5fZGF0ZSwgbWF4X2RhdGUsIGJ5ID0gImRheSIpDQp0cmFkaW5nX2RheXNfaW5kZXggPSAhdHJhZGluZ19kYXlzICVpbiUgb2ZmX2RhdGVzIA0KdHJhZGluZ19kYXlzID0gdHJhZGluZ19kYXlzW3RyYWRpbmdfZGF5c19pbmRleF0NCg0KIyBoZWxwZXIgZnVuY3Rpb24gdG8gZmluZCB0aGUgbmV4dCB0cmFkaW5nIGRheSBhZnRlciB0aGUgY3VycmVudCBkYXRlDQpuZXh0X3RyYWRpbmdfZGF5ID0gZnVuY3Rpb24oY3VycmVudF9kYXRlLCB0cmFkaW5nX2RheXMpIHsNCiAgbmV4dF9kYXkgPSB0cmFkaW5nX2RheXNbdHJhZGluZ19kYXlzID4gY3VycmVudF9kYXRlXQ0KICBpZiAobGVuZ3RoKG5leHRfZGF5KSA+IDApIHsNCiAgICByZXR1cm4obWluKG5leHRfZGF5KSkgICMgcmV0dXJuIHRoZSBuZXh0IGF2YWlsYWJsZSB0cmFkaW5nIGRheQ0KICB9IGVsc2Ugew0KICAgIHJldHVybihOQSkgICMgaWYgbm8gZnV0dXJlIHRyYWRpbmcgZGF5IGlzIGF2YWlsYWJsZSAodW5saWtlbHkgaW4gdGhpcyBjYXNlKQ0KICB9DQp9DQoNCiMgY3JlYXRlIGEgYmluYXJ5IGdvbGQgZGF0YWZyYW1lIHdpdGggdGhlIGxlYWRpbmcgbmV3cyBlZmZlY3QNCmJpbmFyeV9nb2xkX2RmID0gDQogIGFkanVzdGVkX25ld3NfZGYgfD4gDQogIGRwbHlyOjptdXRhdGUoDQogICAgZGF5X29mX3dlZWsgPSBsdWJyaWRhdGU6OndkYXkoZGF0ZSwgbGFiZWwgPSBUUlVFLCBhYmJyID0gRkFMU0UpLA0KICAgIGltcGFjdF9kYXRlID0gZGF0ZSwgICMgc2V0IGltcGFjdF9kYXRlIGFzIG9yaWdpbmFsIGRhdGUgZmlyc3QNCiAgICAjIGlmIGl0J3MgYSBob2xpZGF5IG9yIHdlZWtlbmQsIHBpY2sgdGhlIG5leHQgYXZhaWxhYmxlIHRyYWRpbmcgZGF5DQogICAgaW1wYWN0X2RhdGUgPSBkcGx5cjo6aWZfZWxzZSgNCiAgICAgIGRhdGUgJWluJSBvZmZfZGF0ZXMsIA0KICAgICAgcHVycnI6Om1hcF9kYmwoZGF0ZSwgfiBuZXh0X3RyYWRpbmdfZGF5KC54LCB0cmFkaW5nX2RheXMpKSB8PiBhcy5EYXRlKCksICANCiAgICAgIGltcGFjdF9kYXRlICAjIEtlZXAgb3JpZ2luYWwgZGF0ZSBpZiBub3QgYSB3ZWVrZW5kL2hvbGlkYXkNCiAgICApDQogICkgfD4gDQogICMgbGVmdCBqb2luIHdpdGggcmV0dXJuc19kZiB1c2luZyB0aGUgYWRqdXN0ZWQgaW1wYWN0X2RhdGUNCiAgZHBseXI6OmxlZnRfam9pbigNCiAgICByZXR1cm5zX2RmIHw+IGRwbHlyOjpzZWxlY3QoZGF0ZSwgdGlja2VyLCBsZWFkaW5nX25ld3NfZWZmZWN0KSwNCiAgICBieSA9IGMoImltcGFjdF9kYXRlIiA9ICJkYXRlIiwgInRpY2tlcnMiID0gInRpY2tlciIpDQogICkNCg0KIyBkaXN0cmlidXRpb24gb2YgdGhlIGxlYWRpbmcgbmV3cyBlZmZlY3QNCm5ld3NfZWZmZWN0X3RhYmxlID0gDQogIHRhYmxlKGJpbmFyeV9nb2xkX2RmJGxlYWRpbmdfbmV3c19lZmZlY3QsIHVzZU5BID0gImlmYW55IikgLyAxNA0KDQoNCiMgZXhwbGFuYXRpb24gd2l0aCBjYXQNCmNhdCgNCiAgIlRoZSB0YWJsZSBiZWxvdyBzaG93cyB0aGUgYXZlcmFnZSBkaXN0cmlidXRpb24gb2YgdGhlIGxlYWRpbmcgbmV3cyBlZmZlY3QsIG5vcm1hbGl6ZWQgYnkgdGhlIG51bWJlciBvZiBMTE0gbW9kZWxzICgxNCk6IiwNCiAgIlxuXG5OZWdhdGl2ZSBuZXdzIGVmZmVjdCBjb3VudCBwZXIgbW9kZWw6IiwgbmV3c19lZmZlY3RfdGFibGVbIk5lZ2F0aXZlIl0sDQogICJcblBvc2l0aXZlIG5ld3MgZWZmZWN0IGNvdW50IHBlciBtb2RlbDoiLCBuZXdzX2VmZmVjdF90YWJsZVsiUG9zaXRpdmUiXSwNCiAgIlxuTWlzc2luZyBvciBOQSB2YWx1ZXMgcGVyIG1vZGVsOiIsIG5ld3NfZWZmZWN0X3RhYmxlWzNdLA0KICAiXG5cblRoZSBtaXNzaW5nIG9yIE5BIHZhbHVlcyBhcmUgbGlrZWx5IGR1ZSB0byB0aGUgZmFjdCB0aGF0IHdlIGNvdWxkIG5vdCByZXRyaWV2ZSBzdG9jayBwcmljZXMgZm9yIGNlcnRhaW4gdGlja2VycywgcG9zc2libHkgYmVjYXVzZSB0aGV5IHJlcHJlc2VudCBwcml2YXRlIHN0b2Nrcy4iDQopDQoNCmBgYA0KDQoNCiMjIyBWaXN1YWxpemluZyB0aGUgVmFsaWRpdHkgTWV0cmljcyANCg0KVGhlIGNvbXBhcmlzb24gYWdhaW5zdCBhY3R1YWwgbWFya2V0IGJlaGF2aW9yLCBkZXBpY3RlZCBpbiB0aGUgZmlndXJlIGJlbG93LCBjb250cmFzdHMgdGhlIGJlbmNobWFyayByZXN1bHRzLiBUaGUgZG90IHJlcHJlc2VudHMgZWFjaCBtZXRyaWMncyBtZWFuIHZhbHVlIChmcm9tIGl0cyBpbnRyYS1yYXRlciBmaXZlIHJlcGxpY2F0ZXMpLCBhbmQgdGhlIHdoaXNrZXJzIGxlbmd0aCByZWZsZWN0IHRoZSBzdGFuZGFyZCBlcnJvci4gDQoNCmBgYHtyIHZhbGlkaXR5X1NQNTAwLCBjbGFzcy5zb3VyY2UgPSAnZm9sZC1oaWRlJywgbWVzc2FnZT1GQUxTRSwgZmlnLmFsaWduPSdjZW50ZXInLCBmaWcud2lkdGg9NiwgZmlnLmhlaWdodD04LCBmaWcuZnVsbHdpZHRoPVRSVUV9DQojIGxpYnJhcnkodGlkeXZlcnNlKQ0KIyBsaWJyYXJ5KGdyaWQpDQojIGxpYnJhcnkoZ3JpZEV4dHJhKQ0KDQojIC0tLS0gU2V0dXAgLS0tLQ0KY2hhdF9tb2RlbHMgPSBjKA0KICAicGhpNC1taW5pIiwNCiAgInBoaTQ6bGF0ZXN0IiwNCiAgImxsYW1hMy4yOjFCIiwNCiAgImxsYW1hMy4yOjNCIiwgDQogICJnZW1tYTM6MUIiLA0KICAiZ2VtbWEzOjI3QiIsDQogICJkZWVwc2Vlay1yMToxLjVCIiwNCiAgImRlZXBzZWVrLXIxOjdCIiwNCiAgImNvbW1hbmQtcjdiIiwNCiAgImNvbW1hbmQtci1wbHVzLTA4LTIwMjQiLA0KICAiZ3B0LTRvLW1pbmktMjAyNC0wNy0xOCIsDQogICJncHQtNG8tMjAyNC0xMS0yMCIsDQogICJjbGF1ZGUtMy01LWhhaWt1LTIwMjQxMDIyIiwNCiAgImNsYXVkZS0zLTctc29ubmV0LTIwMjUwMjE5Ig0KKQ0KDQptb2RlbF9hYmJyZXYgPSBjKA0KICAicGhpNC1taW5pIiAgICAgICAgICAgICAgICAgPSAicGhpNC1taW5pIiwNCiAgInBoaTQ6bGF0ZXN0IiAgICAgICAgICAgICAgPSAicGhpNDpsYXRlc3QiLA0KICAibGxhbWEzLjI6MUIiICAgICAgICAgICAgICA9ICJsbGFtYTMuMjoxQiIsDQogICJsbGFtYTMuMjozQiIgICAgICAgICAgICAgID0gImxsYW1hMy4yOjNCIiwNCiAgImdlbW1hMzoxQiIgICAgICAgICAgICAgICAgPSAiZ2VtbWEzOjFCIiwNCiAgImdlbW1hMzoyN0IiICAgICAgICAgICAgICAgPSAiZ2VtbWEzOjI3QiIsDQogICJkZWVwc2Vlay1yMToxLjVCIiAgICAgICAgID0gImRlZXBzZWVrLXIxOjEuNUIiLA0KICAiZGVlcHNlZWstcjE6N0IiICAgICAgICAgICA9ICJkZWVwc2Vlay1yMTo3QiIsDQogICJjb21tYW5kLXI3YiIgICAgICAgICAgICAgID0gImNvbW1hbmQtcjdiIiwNCiAgImNvbW1hbmQtci1wbHVzLTA4LTIwMjQiICAgPSAiY29tbWFuZC1yLXBsdXMiLA0KICAiZ3B0LTRvLW1pbmktMjAyNC0wNy0xOCIgICA9ICJncHQtNG8tbWluaSIsDQogICJncHQtNG8tMjAyNC0xMS0yMCIgICAgICAgID0gImdwdC00byIsDQogICJjbGF1ZGUtMy01LWhhaWt1LTIwMjQxMDIyIj0gImNsYXVkZS0zLTUtaGFpa3UiLA0KICAiY2xhdWRlLTMtNy1zb25uZXQtMjAyNTAyMTkiID0gImNsYXVkZS0zLTctc29ubmV0Ig0KKQ0KDQpjaGF0X21vZGVsX2NvbG9ycyA9IHJlcChjKCcjODA4MDgwJywgJyNDMzE0MkQnKSwgbGVuZ3RoLm91dCA9IGxlbmd0aChjaGF0X21vZGVscykpDQpjaGF0X21vZGVsX2NvbG9ycyA9IHN0YXRzOjpzZXROYW1lcyhjaGF0X21vZGVsX2NvbG9ycywgbW9kZWxfYWJicmV2W2NoYXRfbW9kZWxzXSkNCg0KDQptb2RlbF9sYWJlbHMgPC0gcHVycnI6Om1hcF9jaHIoIG1vZGVsX2FiYnJldltjaGF0X21vZGVsc10sIH57DQogIGlmIChncmVwbCgiXnNwYWNlciIsIC54KSkgcmV0dXJuKCIiKQ0KICBnbHVlOjpnbHVlKCI8c3BhbiBzdHlsZT0nY29sb3I6e2NoYXRfbW9kZWxfY29sb3JzWy54XX0nPjxiPnsueH08L2I+PC9zcGFuPiIpDQp9KQ0KbmFtZXMobW9kZWxfbGFiZWxzKSA8LSBtb2RlbF9hYmJyZXZbY2hhdF9tb2RlbHNdDQoNCg0KDQoNCiMgLS0tLSBMb2FkIERhdGEgLS0tLQ0KaW50ZXJfZGYgPSANCiAgdGliYmxlOjp0cmliYmxlKA0KICAgIH5Nb2RlbCwgICAgICAgICAgICAgfkFjY3VyYWN5LCB+QWNjX1NFLCB+VFBSLCAgICB+VFBSX1NFLCB+VE5SLCAgICB+VE5SX1NFLCB+UFBWLCAgICB+UFBWX1NFLCB+RjEsICAgIH5GMV9TRSwNCiAgICAicGhpNC1taW5pIiwgICAgICAgIDAuNDg3LCAgICAgMC4wMDIsICAgMC40OTUsICAgMC4wMDIsICAgMC40NzksICAgMC4wMDYsICAgMC40ODUsICAgMC4wMDIsICAwLjQ5MCwgICAwLjAwMCwNCiAgICAicGhpNDpsYXRlc3QiLCAgICAgIDAuNTEzLCAgICAgMC4wMDMsICAgMC41MjYsICAgMC4wMDEsICAgMC41MDAsICAgMC4wMDQsICAgMC41MTEsICAgMC4wMDMsICAwLjUxOCwgICAwLjAwMiwNCiAgICAibGxhbWEzLjI6MUIiLCAgICAgIDAuNTEyLCAgICAgMC4wMDAsICAgMC41MjcsICAgMC4wMDEsICAgMC40OTcsICAgMC4wMDEsICAgMC41MTAsICAgMC4wMDAsICAwLjUxOCwgICAwLjAwMCwNCiAgICAibGxhbWEzLjI6M0IiLCAgICAgIDAuNDc4LCAgICAgMC4wMDAsICAgMC41MjUsICAgMC4wMDMsICAgMC40MzIsICAgMC4wMDIsICAgMC40NzgsICAgMC4wMDAsICAwLjUwMSwgICAwLjAwMiwNCiAgICAiZ2VtbWEzOjFCIiwgICAgICAgIDAuNDg5LCAgICAgMC4wMDIsICAgMC40NDIsICAgMC4wMDUsICAgMC41MzUsICAgMC4wMDEsICAgMC40ODYsICAgMC4wMDIsICAwLjQ2MywgICAwLjAwNCwNCiAgICAiZ2VtbWEzOjI3QiIsICAgICAgIDAuNTA5LCAgICAgMC4wMDIsICAgMC41MDUsICAgMC4wMDMsICAgMC41MTIsICAgMC4wMDEsICAgMC41MDcsICAgMC4wMDIsICAwLjUwNiwgICAwLjAwMiwNCiAgICAiZGVlcHNlZWstcjE6MS41QiIsIDAuNDkzLCAgICAgMC4wMDMsICAgMC40NDcsICAgMC4wMTAsICAgMC41MzksICAgMC4wMDQsICAgMC40OTAsICAgMC4wMDMsICAwLjQ2OCwgICAwLjAwNywNCiAgICAiZGVlcHNlZWstcjE6N0IiLCAgIDAuNDk2LCAgICAgMC4wMDIsICAgMC41MDksICAgMC4wMDIsICAgMC40ODMsICAgMC4wMDEsICAgMC40OTQsICAgMC4wMDIsICAwLjUwMSwgICAwLjAwMiwNCiAgICAiY29tbWFuZC1yLXBsdXMiLCAgIDAuNTEyLCAgICAgMC4wMDQsICAgMC40NTAsICAgMC4wMDIsICAgMC41NzQsICAgMC4wMDcsICAgMC41MTEsICAgMC4wMDUsICAwLjQ3OSwgICAwLjAwMywNCiAgICAiY29tbWFuZC1yN2IiLCAgICAgIDAuNTA3LCAgICAgMC4wMDQsICAgMC41NTMsICAgMC4wMDIsICAgMC40NjIsICAgMC4wMDUsICAgMC41MDUsICAgMC4wMDMsICAwLjUyNywgICAwLjAwMywNCiAgICAiZ3B0LTRvLW1pbmkiLCAgICAgIDAuNTAyLCAgICAgMC4wMDIsICAgMC40ODIsICAgMC4wMDQsICAgMC41MjIsICAgMC4wMDMsICAgMC41MDAsICAgMC4wMDMsICAwLjQ5MSwgICAwLjAwMywNCiAgICAiZ3B0LTRvIiwgICAgICAgICAgIDAuNTA5LCAgICAgMC4wMDIsICAgMC40NjEsICAgMC4wMDcsICAgMC41NTgsICAgMC4wMDUsICAgMC41MDgsICAgMC4wMDIsICAwLjQ4MywgICAwLjAwNCwNCiAgICAiY2xhdWRlLTMtNS1oYWlrdSIsIDAuNTEyLCAgICAgMC4wMDMsICAgMC41NDksICAgMC4wMDYsICAgMC40NzYsICAgMC4wMDIsICAgMC41MDksICAgMC4wMDMsICAwLjUyOCwgICAwLjAwNCwNCiAgICAiY2xhdWRlLTMtNy1zb25uZXQiLDAuNTE0LCAgICAgMC4wMDMsICAgMC41NDgsICAgMC4wMDUsICAgMC40ODAsICAgMC4wMDIsICAgMC41MTEsICAgMC4wMDIsICAwLjUyOSwgICAwLjAwNA0KICApDQoNCmludGVyX2RmID0gaW50ZXJfZGYgfD4gDQogIGRwbHlyOjpyZW5hbWUoQWNjdXJhY3lfbWVhbiA9IEFjY3VyYWN5LA0KICAgICAgICAgICAgICAgIFRQUl9tZWFuID0gVFBSLA0KICAgICAgICAgICAgICAgIFROUl9tZWFuID0gVE5SLA0KICAgICAgICAgICAgICAgIFBQVl9tZWFuID0gUFBWLA0KICAgICAgICAgICAgICAgIEYxX21lYW4gPSBGMSkgfD4NCiAgZHBseXI6OnJlbmFtZShBY2N1cmFjeV9zZSA9IEFjY19TRSwNCiAgICAgICAgICAgICAgICBUUFJfc2UgPSBUUFJfU0UsDQogICAgICAgICAgICAgICAgVE5SX3NlID0gVE5SX1NFLA0KICAgICAgICAgICAgICAgIFBQVl9zZSA9IFBQVl9TRSwNCiAgICAgICAgICAgICAgICBGMV9zZSA9IEYxX1NFKQ0KDQoNCmludGVyX2xvbmcgPSBpbnRlcl9kZiB8PiANCiAgdGlkeXI6OnBpdm90X2xvbmdlcigNCiAgICBjb2xzID0gLU1vZGVsLA0KICAgIG5hbWVzX3RvID0gYygiTWV0cmljIiwgIi52YWx1ZSIpLA0KICAgIG5hbWVzX3NlcCA9ICJfIg0KICApIHw+DQogIGRwbHlyOjptdXRhdGUoDQogICAgbG93ZXIgPSBtZWFuIC0gKHNlL3NxcnQoNSkpLA0KICAgIHVwcGVyID0gbWVhbiArIChzZS9zcXJ0KDUpKSwNCiAgICBNb2RlbCA9IGZhY3RvcihNb2RlbCwgbGV2ZWxzID0gcmV2KG1vZGVsX2FiYnJldltjaGF0X21vZGVsc10pKQ0KICAgICkNCg0KdGFibGVfZGYgPC0gaW50ZXJfZGYgfD4NCiAgZHBseXI6OnNlbGVjdChNb2RlbCwgQWNjdXJhY3lfbWVhbiwgVFBSX21lYW4sIFROUl9tZWFuLCBQUFZfbWVhbiwgRjFfbWVhbikgJT4lDQogIGRwbHlyOjpyZW5hbWUoDQogICAgTExNID0gTW9kZWwsDQogICAgQUNDID0gQWNjdXJhY3lfbWVhbiwNCiAgICBUUFIgPSBUUFJfbWVhbiwNCiAgICBUTlIgPSBUTlJfbWVhbiwNCiAgICBQUFYgPSBQUFZfbWVhbiwNCiAgICBGMSA9IEYxX21lYW4pIHw+DQogIGRwbHlyOjptdXRhdGUoZHBseXI6OmFjcm9zcyhkcGx5cjo6d2hlcmUoaXMubnVtZXJpYyksIH4gc3ByaW50ZigiJS4yZiIsIC4pKQ0KICApDQoNCg0KIyAqUGxvdCB0aGUgRGF0YSAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQoNCmludGVyX2xvbmcgfD4gDQogIGdncGxvdDI6OmdncGxvdChnZ3Bsb3QyOjphZXMoeCA9IE1vZGVsLCB5ID0gbWVhbiwgY29sb3IgPSBNb2RlbCkpICsNCiAgZ2dwbG90Mjo6Z2VvbV9ibGFuaygpICsNCiAgZ2dwbG90Mjo6Z2VvbV9wb2ludChzaXplID0gMC45LCBuYS5ybSA9IFRSVUUpICsNCiAgZ2dwbG90Mjo6Z2VvbV9lcnJvcmJhcigNCiAgICBnZ3Bsb3QyOjphZXMoeW1pbiA9IGxvd2VyLCB5bWF4ID0gdXBwZXIpLA0KICAgIHdpZHRoID0gMC40LA0KICAgIG5hLnJtID0gVFJVRQ0KICApICsNCiAgZ2dwbG90Mjo6Z2VvbV90ZXh0KGdncGxvdDI6OmFlcyhsYWJlbCA9IHNwcmludGYoIiUuM2YiLCBsb3dlciksIHkgPSBsb3dlciksIHNpemUgPSAyLjI1LCBoanVzdCA9IDEuMiwgbmEucm0gPSBUUlVFKSArDQogIGdncGxvdDI6Omdlb21fdGV4dChnZ3Bsb3QyOjphZXMobGFiZWwgPSBzcHJpbnRmKCIlLjNmIiwgdXBwZXIpLCB5ID0gdXBwZXIpLCBzaXplID0gMi4yNSwgaGp1c3QgPSAtMC4yLCBuYS5ybSA9IFRSVUUpICsNCiAgZ2dwbG90Mjo6c2NhbGVfY29sb3JfbWFudWFsKHZhbHVlcyA9IGNoYXRfbW9kZWxfY29sb3JzLCBuYS50cmFuc2xhdGUgPSBGQUxTRSkgKw0KICBnZ3Bsb3QyOjpzY2FsZV94X2Rpc2NyZXRlKGxhYmVscyA9IG1vZGVsX2xhYmVscykgKw0KICBnZ3Bsb3QyOjpzY2FsZV95X2NvbnRpbnVvdXMobGltaXRzID0gYygwLjM3NSwgMC42NSksIGJyZWFrcyA9IHNlcSgwLjQsIC42MSwgMC4xKSkgKw0KICBnZ3Bsb3QyOjpjb29yZF9mbGlwKCkgKw0KICBnZ3Bsb3QyOjpmYWNldF93cmFwKH4gTWV0cmljLCBuY29sID0gMiwgc2NhbGVzID0gImZpeGVkIiwgYXhlcyA9ICdhbGwnKSArDQogIGdncGxvdDI6OmxhYnMoDQogICAgdGl0bGUgPSAiQ2xhc3NpZmljYXRpb24gUGVyZm9ybWFuY2UgdnMuIEFjdHVhbCBNYXJrZXQgQmVoYXZpb3IiLA0KICAgIHN1YnRpdGxlID0gIjxzcGFuIHN0eWxlPSdjb2xvcjojODA4MDgwJz5DaGVhcGVyPC9zcGFuPiB2cy4gPHNwYW4gc3R5bGU9J2NvbG9yOiNDMzE0MkQnPm1vcmUgZXhwZW5zaXZlICh0aW1lLCBjb3N0KSA8L3NwYW4+IExMTXMgYnkgY29tcGFueSIsDQogICAgeCA9IE5VTEwsDQogICAgeSA9IE5VTEwNCiAgKSArDQogIGdncGxvdDI6OnRoZW1lX21pbmltYWwoKSArDQogIGdncGxvdDI6OnRoZW1lKA0KICAgIHBsb3QudGl0bGUgPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oaGp1c3QgPSAwLjUsIGZhY2UgPSAiYm9sZCIsIHNpemUgPSA5KSwNCiAgICBwbG90LnN1YnRpdGxlID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGhqdXN0ID0gMC41LCBmYWNlID0gImJvbGQiLCBzaXplID0gOCksDQogICAgYXhpcy50aXRsZS54ID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGhqdXN0ID0gMC41LCBmYWNlID0gImJvbGQiLCBzaXplID0gOCksDQogICAgYXhpcy50aXRsZS55ID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGhqdXN0ID0gMC41LCBmYWNlID0gImJvbGQiLCBzaXplID0gOCksDQogICAgYXhpcy50ZXh0LnggPSBnZ3RleHQ6OmVsZW1lbnRfbWFya2Rvd24oaGp1c3QgPSAwLjUsIHNpemUgPSA3LCBmYWNlID0gJ2JvbGQnKSwNCiAgICBheGlzLnRleHQueSA9IGdndGV4dDo6ZWxlbWVudF9tYXJrZG93bihoanVzdCA9IDEsIHNpemUgPSA3LCBmYWNlID0gJ2JvbGQnKSwNCiAgICBzdHJpcC50ZXh0ID0gZ2d0ZXh0OjplbGVtZW50X21hcmtkb3duKGZhY2UgPSAiYm9sZCIsIHNpemUgPSA4KSwNCiAgICBsZWdlbmQucG9zaXRpb24gPSAibm9uZSIsDQogICAgcGFuZWwuZ3JpZCA9IGdncGxvdDI6OmVsZW1lbnRfYmxhbmsoKSwNCiAgICBheGlzLmxpbmUgPSBnZ3Bsb3QyOjplbGVtZW50X2xpbmUoY29sb3IgPSAiYmxhY2siKSwNCiAgICBheGlzLnRpY2tzID0gZ2dwbG90Mjo6ZWxlbWVudF9saW5lKGNvbG9yID0gJ2JsYWNrJykNCiAgKSAtPiBvcmlnaW5hbF9wbG90DQoNCg0KIyAqIEFkZGluZyBhIHRhYmxlIGluIHRoZSBlbXB0eSBzcGFjZSBhdCB0aGUgYm90dG9tIHJpZ2h0IC0tLS0tLS0tLS0tLS0tLS0tDQoNCiMgQ3VzdG9tIHRoZW1lDQpib29rdGFic190aGVtZSA8LSB0dGhlbWVfbWluaW1hbCgNCiAgY29yZSA9IGxpc3QoDQogICAgZmdfcGFyYW1zID0gbGlzdChmb250ZmFjZSA9ICJwbGFpbiIsIGNleCA9IDAuNTUsIGhqdXN0ID0gMCwgeCA9IDAuMDUpDQogICksDQogIGNvbGhlYWQgPSBsaXN0KA0KICAgIGZnX3BhcmFtcyA9IGxpc3QoZm9udGZhY2UgPSAiYm9sZCIsIGNleCA9IDAuNiwgaGp1c3QgPSAwLCB4ID0gMC4wNSksDQogICAgYmdfcGFyYW1zID0gbGlzdChmaWxsID0gTkEpICAjIE5vIHNoYWRpbmcsIGxpa2UgYm9va3RhYnMNCiAgKSwNCiAgcGFkZGluZyA9IHVuaXQoYygxLjcsIDEuNyksICJtbSIpICAjIFRpZ2h0ZXIgcGFkZGluZw0KKQ0KDQojIEJhc2UgdGFibGUNCnRhYmxlX2dyb2IgPC0gZ3JpZEV4dHJhOjp0YWJsZUdyb2IodGFibGVfZGYsIHJvd3MgPSBOVUxMLCB0aGVtZSA9IGJvb2t0YWJzX3RoZW1lKQ0KdGFibGVfZ3JvYiRoZWlnaHRzIDwtIGdncGxvdDI6OnVuaXQocmVwKDEsIG5yb3codGFibGVfZ3JvYikpLCAibGluZXMiKSAqIDAuNQ0KDQojIEFkZCB0aXRsZSByb3cNCnRpdGxlX2dyb2IgPC0gZ3JpZDo6dGV4dEdyb2IoDQogICJWYWxpZGl0eSB3aXRoIEV4dGVybmFsIENyaXRlcmlvbiIsDQogIGdwID0gZ3JpZDo6Z3Bhcihmb250ZmFjZSA9ICJib2xkIiwgZm9udHNpemUgPSA5KSwNCiAgeCA9IDAuNSwgaGp1c3QgPSAwLjUNCikNCg0KIyBBZGQgdGl0bGUgcm93IGFib3ZlIHRoZSB0YWJsZQ0KdGFibGVfZ3JvYiA8LSBndGFibGU6Omd0YWJsZV9hZGRfcm93cyh0YWJsZV9ncm9iLCBoZWlnaHRzID0gdW5pdCgxLjUsICJsaW5lcyIpLCBwb3MgPSAwKQ0KdGFibGVfZ3JvYiA8LSBndGFibGU6Omd0YWJsZV9hZGRfZ3JvYigNCiAgdGFibGVfZ3JvYiwNCiAgZ3JvYnMgPSB0aXRsZV9ncm9iLA0KICB0ID0gMSwgbCA9IDEsIHIgPSBuY29sKHRhYmxlX2dyb2IpDQopDQoNCnRhYmxlX3dpZHRoIDwtIHN1bSh0YWJsZV9ncm9iJHdpZHRocykNCnRhYmxlX2hlaWdodCA8LSBzdW0odGFibGVfZ3JvYiRoZWlnaHRzKQ0KDQpwYWRkaW5nIDwtIGdncGxvdDI6OnVuaXQoLjUsICJtbSIpDQoNCiMgQ29sb3IgY29kZSB0aGUgdGFibGUgDQpjb3JlX2xsbV9pbmRpY2VzIDwtIHdoaWNoKA0KICB0YWJsZV9ncm9iJGxheW91dCRuYW1lID09ICJjb3JlLWZnIiAmIA0KICAgIHRhYmxlX2dyb2IkbGF5b3V0JGwgPT0gMSAgIyBjb2x1bW4gMQ0KKQ0KDQptb2RlbF9jb2xvcnMgPC0gcmVwKGMoIiM4MDgwODAiLCAiI0MzMTQyRCIpLCBsZW5ndGgub3V0ID0gMTQpICAjIG1hdGNoZXMgaW50ZXJfd2lkZQ0KDQpmb3IgKGkgaW4gc2VxX2Fsb25nKGNvcmVfbGxtX2luZGljZXMpKSB7DQogIGdyb2JfaW5kZXggPC0gY29yZV9sbG1faW5kaWNlc1tpXQ0KICBvcmlnaW5hbF9ncCA8LSB0YWJsZV9ncm9iJGdyb2JzW1tncm9iX2luZGV4XV0kZ3ANCiAgdGFibGVfZ3JvYiRncm9ic1tbZ3JvYl9pbmRleF1dJGdwIDwtIG1vZGlmeUxpc3Qob3JpZ2luYWxfZ3AsIGdwYXIoY29sID0gbW9kZWxfY29sb3JzW2ldKSkNCn0NCg0KIyBBZGQgYm9yZGVyIGFyb3VuZCBlbnRpcmUgdGFibGUgdXNpbmcgZ3JvYlRyZWUNCmJvcmRlcmVkX3RhYmxlIDwtIGdyaWQ6Omdyb2JUcmVlKA0KICBncmlkOjpyZWN0R3JvYigNCiAgICB3aWR0aCA9IHRhYmxlX3dpZHRoKyBwYWRkaW5nLA0KICAgIGhlaWdodCA9IHRhYmxlX2hlaWdodCArIHBhZGRpbmcsDQogICAgZ3AgPSBncmlkOjpncGFyKGZpbGwgPSBOQSwgbHdkID0gMC43LCBjb2wgPSdibGFjaycpIA0KICApLA0KICB0YWJsZV9ncm9iDQopDQoNCg0KZmluYWxfcGxvdCA8LSBjb3dwbG90OjpnZ2RyYXcoKSArDQogIGNvd3Bsb3Q6OmRyYXdfcGxvdChvcmlnaW5hbF9wbG90LCAwLCAwLCAxLCAxKSArICAjIG1haW4gcGxvdCB0YWtlcyBmdWxsIGFyZWENCiAgY293cGxvdDo6ZHJhd19ncm9iKGJvcmRlcmVkX3RhYmxlLCB4ID0gMC43NDIsIHkgPSAwLjE2LCB3aWR0aCA9IDAuMDEsIGhlaWdodCA9IDAuMDEpDQoNCmZpbmFsX3Bsb3QNCg0KYGBg

Evaluating the Consistency of Multiple LLMs with Intra and Inter Reliability Methods

Fadel M. Megahed 1

Ying-Ju (Tessa) Chen 2

Allison Jones-Farmer 3

Gabe Lee 4

Brooke Wang 5

Inez M. Zwetsloot 6

May 20, 2025

Preface

1 Objectives of this Analysis

2 Sample Size Calculations for our Binary Classification Experiment

3 Experimental Setup for the Consistency Analysis

3.1 LLMs Used and their API Keys

3.2 A Custom Function for LLM Calls

3.3 A Custom Function for Extracting the LLM Labels

3.4 Custom Functions for Computing the Reliability Metrics

3.5 Custom Functions to Compute Agreement with Benchmark

4 The Binary Classification Experiment

4.1 Extracting the Full Dataset

4.2 Downsampling the Binary Classification Dataset

4.3 LLM-based Labeling

4.3.1 Prompt Construction

4.3.2 Labeling the Binary Classification Dataset

4.3.3 Task Completion Time by Model

4.3.4 Extracting the LLM Labels and Creating a Data Frame for Analysis

4.3.5 Distribution of LLMs for Binary Classification

4.4 Reliability Assessment

4.4.1 Distribution of NA-Penalized Agreement Across Articles by LLM

4.4.2 Computing the Reliability Metrics

4.4.3 Intra-LLM Reliability

4.4.4 Inter-Rater Reliability among LLMs

4.5 Validity of the Binary Classification Labels (Comparison with “StockNewsAPI” Labels)

4.5.1 Visualizing the Validity Metrics

4.6 Validity of the Binary Classification Labels (Comparison with “External Criterion” Impacts)

4.6.1 Extracting the External Criterion Stock Performance

4.6.2 Computing the Leading News Effect

4.6.3 Merging the News Effect with Our Binary Classification Labels

4.6.4 Visualizing the Validity Metrics

Fadel M. Megahed ¹

Ying-Ju (Tessa) Chen ²

Allison Jones-Farmer ³

Gabe Lee ⁴

Brooke Wang ⁵

Inez M. Zwetsloot ⁶

4.6.3 Merging the `News Effect` with Our Binary Classification Labels