Gene-Ontologies and Annotation

Gene-Ontology Analysis

In the early days of microarray analysis, people were happy if they got a handful of differentially-expressed genes that they could validate or follow-up. However, with later technologies (and depending on the experimental setup) we might have thousands of statistically-significant results, which no-one has the time to follow-up. Also, we might be interested in pathways / mechanisms that are altered and not just individual genes.

In this section we move towards discovering if our results are biologically significant. Are the genes that we have picked statistical flukes, or are there some commonalities.

There are two different approaches one might use, and we will cover the theory behind both

  • There is also a bunch of websites for doing the tests
    • we will show how they are done in Bioconductor so the theory is clear
  • We will assume we have done a differential-expression analysis, but the same techniques can be used for other situations when we have a gene list
    • ChIP-seq
    • RNA-seq

Theory Part I: Over-representation analysis

  • “Threshold-based”: require defintion of a statistical threshold to define list of genes to test (e.g. FDR < 0.01)
  • Hypergeometric test or Fisher’s Exact test generally used.

The question we are asking here is;

“Are the number of DE genes associated with Theme X significantly greater than what we might expect by chance alone?”

Where Theme X could be genes belonging to a particular GO (Gene Onotology) term.

Let’s imagine that we have a bag full of balls. Each balls represents a gene in the gene universe. - Paint the balls representing our selected list grey, and paint the rest red.

In this small example, we can define;

  • Total number of balls: 40
  • Total number of interesting (grey) balls: 10

Now, lets select a number (say, 12) of the balls at random without seeing into the bag and look at what we get

We have picked, at random, 8 grey balls. Using simulations, we can repeat the process and look at how many grey we get. We can (of course!) do this in R. In base R there is a family of functions that can generate random draws from various distributions; rnorm, rbinom, rhyper etc….

The distribution of the data shows what the most-likely values are

#see ?rhyper for argument definition
trials <- rhyper(10000,40,10,12)
hist(trials)

We can count how many times each value is observed

table(trials)
trials
   5    6    7    8    9   10   11   12 
   7   65  403 1333 2652 3111 1924  505 

Dividing by the number of trials gives a probability of sorts

table(trials)/10000
trials
     5      6      7      8      9     10     11     12 
0.0007 0.0065 0.0403 0.1333 0.2652 0.3111 0.1924 0.0505 

The probability of getting at least a certain number can also be computed

cumsum(table(trials)/10000)
     5      6      7      8      9     10     11     12 
0.0007 0.0072 0.0475 0.1808 0.4460 0.7571 0.9495 1.0000 
1-cumsum(table(trials)/10000)
     5      6      7      8      9     10     11     12 
0.9993 0.9928 0.9525 0.8192 0.5540 0.2429 0.0505 0.0000 

Back to our example, the distribution of balls can be expressed as a contingency table, on which we can use a Fisher’s exact test

Total grey balls: 10 Total in subset: 12

df <- data.frame(Selection_in = c(8,4), Selection_out = c(2,26))
rownames(df) <- c("Grey_in", "Grey_out")
df
df <- data.frame(Selection_in = c("a","c","a+c"), Selection_out = c("b","d","b+d"), RowTotal = c("a +b","c+d","a+b+c+d (=n)"))
rownames(df) <- c("Grey_in", "Grey_out","Column Total")
df

The formula for Fishers exact test is;

\[ p = \frac{\binom{a + b}{a}\binom{c +d}{c}}{\binom{n}{a +c}} = \frac{(a+b)!(c+d)!(a+c)!(b+d)!}{a!b!c!d!n!} \]

or less formally;

P = (ways of choosing grey balls) X (ways of non-grey balls amongst subset) / ways of choosing subset

Be careful of how you define the universe

What genes were candidates for selection as universe?

  • All possible genes
  • All genes represented on the chip
  • All genes that have a GO annotation
  • All genes from the chip that pass a non-specific filter

Just changing the size of the universe alone can have a massive effect on the p-value

  • In the formula, we have to divide by the universe size
    • so the probability will become very small as this increases
    • even if all the other numbers stay the same
## Code from Seth Falcon
P <- function(size){
  nFound <- 10
  nDrawn <- 400
  nAtCat <- 40
  nNotAtCat <- size - nAtCat
  phyper(nFound-1, nAtCat,nNotAtCat,nDrawn,lower.tail = FALSE)
}
P(1000)
[1] 0.9860212
P(5000)
[1] 0.0009145082

Preparing the data for an over-representation test

As before, we download the data from GEO.

library(GEOquery)
library(genefilter)
url <- "ftp://ftp.ncbi.nih.gov/pub/geo/DATA/SeriesMatrix/GSE33126/GSE33126_series_matrix.txt.gz"
filenm <- "data/GSE33126_series_matrix.txt.gz"
if(!file.exists("data/GSE33126_series_matrix.txt.gz")) download.file(url, destfile=filenm)
colonData <- getGEO(filename=filenm)
File stored at: 
/tmp/RtmpTk1Ym1/GPL6947.soft
not all columns named in 'colClasses' exist
colonData
ExpressionSet (storageMode: lockedEnvironment)
assayData: 48803 features, 18 samples 
  element names: exprs 
protocolData: none
phenoData
  sampleNames: GSM820516 GSM820517 ... GSM820533 (18 total)
  varLabels: title geo_accession ... data_row_count (31 total)
  varMetadata: labelDescription
featureData
  featureNames: ILMN_1343291 ILMN_1343295 ... ILMN_2416019 (48803 total)
  fvarLabels: ID nuID ... GB_ACC (30 total)
  fvarMetadata: Column Description labelDescription
experimentData: use 'experimentData(object)'
Annotation: GPL6947 

Remember that we also need to do the log\(_2\) transformation

exprs(colonData) <- log2(exprs(colonData))

We are now going to create a gene universe by removing genes for will not contribute to the subsequent analysis. Such filtering is done without regarding the phenotype variables - hence a ”non-specific” filter. An Illumina Human6 chip contains around 48,00 probes, but less than half of these have enough detailed information to useful for a GO analysis. Therefore we restrict the dataset to only probes for which we have a Entrez ID. Moreover, we would like only the most-variable probe for a given gene.

  • This can be performed automatically using the nsFilter function
  • First we have to tell the function we are dealing with illuminaHumanv3 data
  • The output of nsFilter contains several components
    • we want just the expression set of filtered rows, so we need to select this
library(genefilter)
annotation(colonData) <- "illuminaHumanv3"
filt <- nsFilter(colonData)

Attaching package: ‘S4Vectors’

The following objects are masked from ‘package:base’:

    colMeans, colSums, expand.grid, rowMeans, rowSums
colonData.filt <- filt$eset

We’ll write-out the names of the Entrez genes that we have so we can use this as our Gene Universe.

anno.final <- fData(colonData.filt)
universeIds <- anno.final$Entrez_Gene_ID
length(universeIds)
[1] 9651
write.table(universeIds, file="geneUniverse.txt",sep="\t",row.names = FALSE,quote=FALSE)

We’ve previously explained the following steps to produce a table of DE genes. However, for this analysis we are interested in a relatively long gene-list and do not need to be so stringent, so we can apply a standard t-test

  • The rowttests function is another useful function for applying the same operation to the rows of a matrix, in this case a t-test
library(genefilter)
SampleGroup <- pData(colonData)$source_name_ch1
Patient <- pData(colonData)$characteristics_ch1.1
ttests <- rowttests(colonData.filt, fac=SampleGroup)
cutoff <- 0.01
selected <- ttests$p.value < cutoff
selectedEntrezIds <- fData(colonData.filt)$Entrez_Gene_ID[selected]
length(selectedEntrezIds)
[1] 2226

Using online tools

Even though this is a course advocating the use of R and Bioconductor, there are in fact many online tools that can assist with gene set analyses. DAVID is a very popular tool and is able to accept gene lists in the form of Entrez IDs.

We write the selected Entrez IDs to a text file. So that the file is recognised by DAVID, we need to specify the argument row.names=FALSE to make sure that row numbers do not appear in the file.

write.table(selectedEntrezIds, file="myEntrezGene.txt",sep="\t",quote=FALSE,row.names = FALSE)

You can upload this text file to the DAVID website, along with the gene universe we created previously, and follow the instructions.

Over-representation analysis in Bioconductor: GOstats

The hyperGTest function is used to do the hypergeometric test for GO terms. Rather than passing a long list of parameters to the function. An object of type GOHyperGParams is created to hold all the parameters we need to run the hypergeometric test. This object can then be passed to hyperGTest multiple times without having to re-type the parameters each time. The meanings of these parameters are as follows:

  • geneIds - The list of identifiers for the genes that we have selected as interesting
  • universeGeneIds - The list of identifiers resulting from non-specific filtering
  • annotation - The name of the annotation package that will be used
  • ontology - The name of the GO ontology that will be tested; either BP, CC or MF
  • pvaluecutoff - p-value that we will use to select significant GO terms
  • testDirection - Either ”over” or ”under” for over or under represented terms respectively
  • conditional - A more sophisticated form of hypergeometric test, which takes the relationships between terms in the GO graph can be used if this is set to TRUE. For this practical we will keep conditional = FALSE
library(GOstats)
Loading required package: Category
Loading required package: Matrix

Attaching package: ‘Matrix’

The following object is masked from ‘package:S4Vectors’:

    expand

Loading required package: graph


Attaching package: ‘GOstats’

The following object is masked from ‘package:AnnotationDbi’:

    makeGOGraph
params = new ("GOHyperGParams" , geneIds = selectedEntrezIds , 
              universeGeneIds = universeIds , annotation = "illuminaHumanv3" ,
              ontology =  "BP" , pvalueCutoff = 0.05 , conditional = FALSE ,
              testDirection = "over")
removing duplicate IDs in geneIdsremoving duplicate IDs in universeGeneIds
hgOver = hyperGTest(params)
hgOver
Gene to GO BP  test for over-representation 
9154 GO BP ids tested (674 have p < 0.05)
Selected gene set size: 1948 
    Gene universe size: 8292 
    Annotation package: illuminaHumanv3 

The summary function can be used to view the results of the test in matrix form. The rows of the matrix are arranged in order of significance. The p-value is shown for each GO term along with with total number of genes for that GO term, number of genes we would be ex- pect to appear in the gene list by chance and that number that were observed. A descriptive name is also given for each term. The results can also be printed out to a HTML report using the htmlReport function.

summary (hgOver)[1:20 , ]
htmlReport(hgOver,file="go-summary.html")

Theory Part II: Threshold-free

For these tests, we don’t have to specify a statistical threshold and use the test statistics from all genes as the input to the test. The popular Gene Set Enrichment Analysis (GSEA) uses this approach. These tests can be used to detect differential expression for a group of genes, even when the effects are too small or there is too little data to detect the genes individually.

gsea

gsea

Subramanian et al, PNAS 2005

The Broad institute provides a version of GSEA that can be run via a java application. However, it requires you to convert your data into a particular format. A more convenient option is GeneTrail which has the option to do a GSEA-style analysis.

The required input text contains a set of gene identifiers that are ordered according to some test statistic. To produce such a file, we first have to order the t-statistics from our test ttests$statistic with the order function.

gseaList <- anno.final$Entrez_Gene_ID[order(ttests$statistic,decreasing = TRUE)]
write.table(gseaList, file="genetrail-gsea.txt",sep="\t",row.names = FALSE,quote=FALSE)

Note that Genetrail can also be used for the Over-Representation analysis that we saw in the previous section.

A Threshold-free test in Bioconductor

A simple version of this test is implemented in the geneSetTest function in limma, which computes a p-value to test the hypothesis that the selected genes have more extreme test-statistics than one might expect by chance. Moreover, separate tests can be performed to see if the selected genes are up-regulated (alternative=up) or down-regulated (alternative=down). The default is to test for extreme statistics regardless of sign.

To demonstrate, we will pick 50 genes at random. We have to supply the entire set of test statistics along indices of interest. The result of the barcode plot below shows there is no particular trend for genes we have picked be up- or down-regulated.

  • Here sample picks a random set of n values from a given vector
  • As we expect, the random rows are neither skewed towards to top or bottom of the ranked list
library (limma)
randGenes <- sample(1:nrow(ttests),50)
randGenes
 [1] 7210 5228 8544 1904 6311 3421 6892 8376 4695   83  376 2467 4213 9312 3621 1334 8950 3420 3131 6897 4887 9507 4753 3431 2224
[26]  318 2182 1604 9580 9553 3792 5331 2499 6917 4444 1209 4086 5893 6454 1314 8974 1971 4393 5104 2505 4762 1294 5514 3076 7852
geneSetTest (index = randGenes , statistics = ttests$statistic)
[1] 0.5521598
geneSetTest (index = randGenes , statistics = ttests$statistic ,
  alternative = "down" )
[1] 0.4267091
geneSetTest (index = randGenes , statistics = ttests$statistic ,
  alternative = "up" )
[1] 0.5733109

A useful way of visualising the results is with a barcodeplot. The statistics are ranked left to right from largest to smallest. The ranked statistics are represented by a shaded bar or bed, and the positions of the specified subsets are marked by vertical bars, forming a pattern like a barcode. Inspired by GSEA, the line graph above the plot (an enrichment worm) gives a sense of whereabouts the enrichment is highest.

barcodeplot (statistics = as.numeric(ttests$statistic) , index = randGenes )

On the other hand, we could deliberately pick genes with a significant p-value and notice how the results are altered.

myGenes <- which(ttests$p.value < 0.01)[1:50]
geneSetTest (index = myGenes , statistics = ttests$statistic)
[1] 9.906015e-22
geneSetTest (index = myGenes , statistics = ttests$statistic ,
  alternative = "down" )
[1] 0.1738758
geneSetTest (index = myGenes , statistics = ttests$statistic ,
  alternative = "up" )
[1] 0.8261373
barcodeplot (statistics = as.numeric(ttests$statistic) , index = myGenes )

If we have a specific query about a set of genes, we need to locate them in the list of ranked statistics.

  • We have already seen of example of searching for particular gene symbols in our output tables.
  • We were careful to make sure the annotation was in the same order as the expression matrix, so this should be straightforward
mylist <- c("LOC441066","ARF3","FMNL3","CSF1R","XLKD1","TTRAP","DMWD","SYNPO2L","PILRB","LAMP3")
myGenes <- which(anno.final$Symbol %in% mylist)
geneSetTest (index = myGenes , statistics = ttests$statistic)
[1] 0.998012
geneSetTest (index = myGenes , statistics = ttests$statistic ,
  alternative = "down" )
[1] 0.6866257
geneSetTest (index = myGenes , statistics = ttests$statistic ,
  alternative = "up" )
[1] 0.3134224
barcodeplot (statistics = as.numeric(ttests$statistic) , index = myGenes)

However, if we want to interrogate the results for a particular GO or KEGG pathway we could look up the identifiers manually, or use some useful functionality in Bioconductor.

Annotation

Converting between different identifiers

There are facilities in Bioconductor for converting between different identifier (e.g. Entrez, Ensembl, Unigene) schemes. For each organism, there is a pre-built database package that allows various queries to be made. In this example, we will use the org.Hs.eg.db package for humans. Other organisms can be found on the list of Bioconductor annotation packages; just look for packages that start org..

An advantage of using these packages, rather than web services such as biomart, is that offline queries are possible.

Each organism has a series of columns and keys defined. The keys are the identifers that you want to query, and the columns define the information that you want to retrieve.

library(org.Hs.eg.db)
columns(org.Hs.eg.db)
 [1] "ACCNUM"       "ALIAS"        "ENSEMBL"      "ENSEMBLPROT"  "ENSEMBLTRANS" "ENTREZID"     "ENZYME"       "EVIDENCE"    
 [9] "EVIDENCEALL"  "GENENAME"     "GO"           "GOALL"        "IPI"          "MAP"          "OMIM"         "ONTOLOGY"    
[17] "ONTOLOGYALL"  "PATH"         "PFAM"         "PMID"         "PROSITE"      "REFSEQ"       "SYMBOL"       "UCSCKG"      
[25] "UNIGENE"      "UNIPROT"     
keytypes(org.Hs.eg.db)
 [1] "ACCNUM"       "ALIAS"        "ENSEMBL"      "ENSEMBLPROT"  "ENSEMBLTRANS" "ENTREZID"     "ENZYME"       "EVIDENCE"    
 [9] "EVIDENCEALL"  "GENENAME"     "GO"           "GOALL"        "IPI"          "MAP"          "OMIM"         "ONTOLOGY"    
[17] "ONTOLOGYALL"  "PATH"         "PFAM"         "PMID"         "PROSITE"      "REFSEQ"       "SYMBOL"       "UCSCKG"      
[25] "UNIGENE"      "UNIPROT"     

Lets’ find out what the Entrez ID’s are for the genes TP53, BRCA1 and PTEN. We specify our keys as the vector of these names, our keytype is SYMBOL (which must be in the result we get back from doing keytypes) and the columns is ENTREZID.

select(org.Hs.eg.db, keys=c("TP53","BRCA1","PTEN"), keytype = "SYMBOL",columns = "ENTREZID")
'select()' returned 1:1 mapping between keys and columns

We can choose to return multiple bits of information by changing the columns argument. In some cases though, you could get many results returned for the same key.

select(org.Hs.eg.db, keys=c("TP53","BRCA1","PTEN"), keytype = "SYMBOL",columns = c("ENTREZID","ENSEMBL","UNIPROT"))
'select()' returned 1:many mapping between keys and columns

We can find out which genes belong to a particular path. e.g. the Cell Cycle pathway is KEGG: 04110 and the keytype is PATH.

select(org.Hs.eg.db, keys="04110", keytype = "PATH",columns=c("SYMBOL","ENTREZID"))
'select()' returned 1:many mapping between keys and columns

Gene-Ontology terms can be queried using the GOALL keytype. Also returned in the result are the GO evidence codes and ontology type (BP = Biological Process, MF = Molecular Function).

For example, we might be interested in the GO term *GO:0016072* and be interested in whether these genes are up- or down-regulated in our study. The first step would be to retrieve the Entrez IDs that correspond to this term.

goQuery <- select(org.Hs.eg.db, keys="GO:0016072", keytype = "GOALL",columns=c("SYMBOL","ENTREZID"))
'select()' returned 1:many mapping between keys and columns
head(goQuery)

We can match these Entrez IDs to our annotation object and proceed as before.

myGenes <- which(anno.final$Entrez %in% goQuery$ENTREZ)
geneSetTest (index = myGenes , statistics = ttests$statistic)
[1] 3.761809e-11
barcodeplot (statistics = as.numeric(ttests$statistic) , index = myGenes )

LS0tCnRpdGxlOiAiT250b2xvZ2llcyBhbmQgRW5yaWNobWVudCIKYXV0aG9yOiAiTWFyayBEdW5uaW5nOyBtYXJrICdkb3QnIGR1bm5pbmcgJ2F0JyBjcnVrLmNhbS5hYy51ayIKZGF0ZTogJ2ByIGZvcm1hdChTeXMudGltZSgpLCAiTGFzdCBtb2RpZmllZDogJWQgJWIgJVkiKWAnCm91dHB1dDogCiAgaHRtbF9ub3RlYm9vazogCiAgICB0b2M6IHllcwogICAgdG9jX2Zsb2F0OiB5ZXMKLS0tCmBgYHtyIGVjaG89RkFMU0UsbWVzc2FnZT1GQUxTRX0KbGlicmFyeShHRU9xdWVyeSkKbGlicmFyeShsaW1tYSkKYGBgCgojIEdlbmUtT250b2xvZ2llcyBhbmQgQW5ub3RhdGlvbgoKIyMgR2VuZS1PbnRvbG9neSBBbmFseXNpcwoKSW4gdGhlIGVhcmx5IGRheXMgb2YgbWljcm9hcnJheSBhbmFseXNpcywgcGVvcGxlIHdlcmUgaGFwcHkgaWYgdGhleSBnb3QgYSBoYW5kZnVsIG9mIGRpZmZlcmVudGlhbGx5LWV4cHJlc3NlZCBnZW5lcyB0aGF0IHRoZXkgY291bGQgdmFsaWRhdGUgb3IgZm9sbG93LXVwLiBIb3dldmVyLCB3aXRoIGxhdGVyIHRlY2hub2xvZ2llcyAoYW5kIGRlcGVuZGluZyBvbiB0aGUgZXhwZXJpbWVudGFsIHNldHVwKSB3ZSBtaWdodCBoYXZlIHRob3VzYW5kcyBvZiBzdGF0aXN0aWNhbGx5LXNpZ25pZmljYW50IHJlc3VsdHMsIHdoaWNoIG5vLW9uZSBoYXMgdGhlIHRpbWUgdG8gZm9sbG93LXVwLiBBbHNvLCB3ZSBtaWdodCBiZSBpbnRlcmVzdGVkIGluIHBhdGh3YXlzIC8gbWVjaGFuaXNtcyB0aGF0IGFyZSBhbHRlcmVkIGFuZCBub3QganVzdCBpbmRpdmlkdWFsIGdlbmVzLgoKSW4gdGhpcyBzZWN0aW9uIHdlIG1vdmUgdG93YXJkcyBkaXNjb3ZlcmluZyBpZiBvdXIgcmVzdWx0cyBhcmUgKioqYmlvbG9naWNhbGx5IHNpZ25pZmljYW50KioqLiBBcmUgdGhlIGdlbmVzIHRoYXQgd2UgaGF2ZSBwaWNrZWQgc3RhdGlzdGljYWwgZmx1a2VzLCBvciBhcmUgdGhlcmUgc29tZSBjb21tb25hbGl0aWVzLiAKClRoZXJlIGFyZSB0d28gZGlmZmVyZW50IGFwcHJvYWNoZXMgb25lIG1pZ2h0IHVzZSwgYW5kIHdlIHdpbGwgY292ZXIgdGhlIHRoZW9yeSBiZWhpbmQgYm90aAoKLSBUaGVyZSBpcyBhbHNvIGEgYnVuY2ggb2Ygd2Vic2l0ZXMgZm9yIGRvaW5nIHRoZSB0ZXN0cwogICAgKyB3ZSB3aWxsIHNob3cgaG93IHRoZXkgYXJlIGRvbmUgaW4gQmlvY29uZHVjdG9yIHNvIHRoZSB0aGVvcnkgaXMgY2xlYXIKLSBXZSB3aWxsIGFzc3VtZSB3ZSBoYXZlIGRvbmUgYSBkaWZmZXJlbnRpYWwtZXhwcmVzc2lvbiBhbmFseXNpcywgYnV0IHRoZSBzYW1lIHRlY2huaXF1ZXMgY2FuIGJlIHVzZWQgZm9yIG90aGVyIHNpdHVhdGlvbnMgd2hlbiB3ZSBoYXZlIGEgZ2VuZSBsaXN0CiAgICArIENoSVAtc2VxCiAgICArIFJOQS1zZXEKICAgIAoKIyMgVGhlb3J5IFBhcnQgSTogT3Zlci1yZXByZXNlbnRhdGlvbiBhbmFseXNpcwoKLSAiVGhyZXNob2xkLWJhc2VkIjogcmVxdWlyZSBkZWZpbnRpb24gb2YgYSBzdGF0aXN0aWNhbCB0aHJlc2hvbGQgdG8gZGVmaW5lIGxpc3Qgb2YgZ2VuZXMgdG8gdGVzdCAoZS5nLiBGRFIgPCAwLjAxKQotIEh5cGVyZ2VvbWV0cmljIHRlc3Qgb3IgRmlzaGVyJ3MgRXhhY3QgdGVzdCBnZW5lcmFsbHkgdXNlZC4KClRoZSBxdWVzdGlvbiB3ZSBhcmUgYXNraW5nIGhlcmUgaXM7Cgo+ICoqKiJBcmUgdGhlIG51bWJlciBvZiBERSBnZW5lcyBhc3NvY2lhdGVkIHdpdGggVGhlbWUgWCBzaWduaWZpY2FudGx5IGdyZWF0ZXIgdGhhbiB3aGF0IHdlIG1pZ2h0IGV4cGVjdCBieSBjaGFuY2UgYWxvbmU/IioqKgoKV2hlcmUgVGhlbWUgWCBjb3VsZCBiZSBnZW5lcyBiZWxvbmdpbmcgdG8gYSBwYXJ0aWN1bGFyIEdPIChHZW5lIE9ub3RvbG9neSkgdGVybS4KCkxldCdzIGltYWdpbmUgdGhhdCB3ZSBoYXZlIGEgYmFnIGZ1bGwgb2YgYmFsbHMuIEVhY2ggYmFsbHMgcmVwcmVzZW50cyBhIGdlbmUgaW4gdGhlICpnZW5lIHVuaXZlcnNlKi4gCi0gUGFpbnQgdGhlIGJhbGxzIHJlcHJlc2VudGluZyBvdXIgc2VsZWN0ZWQgbGlzdCBncmV5LCBhbmQgcGFpbnQgdGhlIHJlc3QgcmVkLgoKCiFbXShpbWFnZXMvYmFnLWFuZC1iYWxscy5wbmcpCgpJbiB0aGlzIHNtYWxsIGV4YW1wbGUsIHdlIGNhbiBkZWZpbmU7CgotIFRvdGFsIG51bWJlciBvZiBiYWxsczogNDAKLSBUb3RhbCBudW1iZXIgb2YgaW50ZXJlc3RpbmcgKGdyZXkpIGJhbGxzOiAxMAoKTm93LCBsZXRzIHNlbGVjdCBhIG51bWJlciAoc2F5LCAxMikgb2YgdGhlIGJhbGxzIGF0IHJhbmRvbSB3aXRob3V0IHNlZWluZyBpbnRvIHRoZSBiYWcgYW5kIGxvb2sgYXQgd2hhdCB3ZSBnZXQKCiFbXShpbWFnZXMvcGlja2VkLWJhbGxzLnBuZykKCgpXZSBoYXZlIHBpY2tlZCwgYXQgcmFuZG9tLCA4IGdyZXkgYmFsbHMuIFVzaW5nIHNpbXVsYXRpb25zLCB3ZSBjYW4gcmVwZWF0IHRoZSBwcm9jZXNzIGFuZCBsb29rIGF0IGhvdyBtYW55IGdyZXkgd2UgZ2V0LiBXZSBjYW4gKG9mIGNvdXJzZSEpIGRvIHRoaXMgaW4gUi4gSW4gYmFzZSBSIHRoZXJlIGlzIGEgZmFtaWx5IG9mIGZ1bmN0aW9ucyB0aGF0IGNhbiBnZW5lcmF0ZSByYW5kb20gZHJhd3MgZnJvbSB2YXJpb3VzIGRpc3RyaWJ1dGlvbnM7IGBybm9ybWAsIGByYmlub21gLCBgcmh5cGVyYCBldGMuLi4uCgpUaGUgZGlzdHJpYnV0aW9uIG9mIHRoZSBkYXRhIHNob3dzIHdoYXQgdGhlIG1vc3QtbGlrZWx5IHZhbHVlcyBhcmUKCmBgYHtyfQojc2VlID9yaHlwZXIgZm9yIGFyZ3VtZW50IGRlZmluaXRpb24KdHJpYWxzIDwtIHJoeXBlcigxMDAwMCw0MCwxMCwxMikKaGlzdCh0cmlhbHMpCmBgYAoKV2UgY2FuIGNvdW50IGhvdyBtYW55IHRpbWVzIGVhY2ggdmFsdWUgaXMgb2JzZXJ2ZWQKCmBgYHtyfQp0YWJsZSh0cmlhbHMpCmBgYAoKRGl2aWRpbmcgYnkgdGhlIG51bWJlciBvZiB0cmlhbHMgZ2l2ZXMgYSBwcm9iYWJpbGl0eSBvZiBzb3J0cwoKYGBge3J9CnRhYmxlKHRyaWFscykvMTAwMDAKCmBgYAoKVGhlIHByb2JhYmlsaXR5IG9mIGdldHRpbmcgKmF0IGxlYXN0KiBhIGNlcnRhaW4gbnVtYmVyIGNhbiBhbHNvIGJlIGNvbXB1dGVkCgpgYGB7cn0KY3Vtc3VtKHRhYmxlKHRyaWFscykvMTAwMDApCjEtY3Vtc3VtKHRhYmxlKHRyaWFscykvMTAwMDApCmBgYAoKCgpCYWNrIHRvIG91ciBleGFtcGxlLCB0aGUgZGlzdHJpYnV0aW9uIG9mIGJhbGxzIGNhbiBiZSBleHByZXNzZWQgYXMgYSBjb250aW5nZW5jeSB0YWJsZSwgb24gd2hpY2ggd2UgY2FuIHVzZSBhIEZpc2hlcidzIGV4YWN0IHRlc3QKClRvdGFsIGdyZXkgYmFsbHM6IDEwClRvdGFsIGluIHN1YnNldDogMTIKCmBgYHtyfQpkZiA8LSBkYXRhLmZyYW1lKFNlbGVjdGlvbl9pbiA9IGMoOCw0KSwgU2VsZWN0aW9uX291dCA9IGMoMiwyNikpCnJvd25hbWVzKGRmKSA8LSBjKCJHcmV5X2luIiwgIkdyZXlfb3V0IikKZGYKYGBgCgoKCmBgYHtyfQpkZiA8LSBkYXRhLmZyYW1lKFNlbGVjdGlvbl9pbiA9IGMoImEiLCJjIiwiYStjIiksIFNlbGVjdGlvbl9vdXQgPSBjKCJiIiwiZCIsImIrZCIpLCBSb3dUb3RhbCA9IGMoImEgK2IiLCJjK2QiLCJhK2IrYytkICg9bikiKSkKcm93bmFtZXMoZGYpIDwtIGMoIkdyZXlfaW4iLCAiR3JleV9vdXQiLCJDb2x1bW4gVG90YWwiKQpkZgpgYGAKClRoZSBmb3JtdWxhIGZvciBGaXNoZXJzIGV4YWN0IHRlc3QgaXM7CgokJCBwID0gXGZyYWN7XGJpbm9te2EgKyBifXthfVxiaW5vbXtjICtkfXtjfX17XGJpbm9te259e2EgK2N9fSA9IFxmcmFjeyhhK2IpIShjK2QpIShhK2MpIShiK2QpIX17YSFiIWMhZCFuIX0gJCQKCm9yIGxlc3MgZm9ybWFsbHk7CgoqUCA9ICh3YXlzIG9mIGNob29zaW5nIGdyZXkgYmFsbHMpIFggKHdheXMgb2Ygbm9uLWdyZXkgYmFsbHMgYW1vbmdzdCBzdWJzZXQpIC8gd2F5cyBvZiBjaG9vc2luZyBzdWJzZXQqCgoKKipCZSBjYXJlZnVsIG9mIGhvdyB5b3UgZGVmaW5lIHRoZSB1bml2ZXJzZSoqCgpXaGF0IGdlbmVzIHdlcmUgY2FuZGlkYXRlcyBmb3Igc2VsZWN0aW9uIGFzIHVuaXZlcnNlPwoKLSBBbGwgcG9zc2libGUgZ2VuZXMKLSBBbGwgZ2VuZXMgcmVwcmVzZW50ZWQgb24gdGhlIGNoaXAKLSBBbGwgZ2VuZXMgdGhhdCBoYXZlIGEgR08gYW5ub3RhdGlvbgotIEFsbCBnZW5lcyBmcm9tIHRoZSBjaGlwIHRoYXQgcGFzcyBhICoqbm9uLXNwZWNpZmljKiogZmlsdGVyCgpKdXN0IGNoYW5naW5nIHRoZSBzaXplIG9mIHRoZSB1bml2ZXJzZSBhbG9uZSBjYW4gaGF2ZSBhIG1hc3NpdmUgZWZmZWN0IG9uIHRoZSBwLXZhbHVlCgotIEluIHRoZSBmb3JtdWxhLCB3ZSBoYXZlIHRvIGRpdmlkZSBieSB0aGUgdW5pdmVyc2Ugc2l6ZQogICAgKyBzbyB0aGUgcHJvYmFiaWxpdHkgd2lsbCBiZWNvbWUgdmVyeSBzbWFsbCBhcyB0aGlzIGluY3JlYXNlcwogICAgKyBldmVuIGlmIGFsbCB0aGUgb3RoZXIgbnVtYmVycyBzdGF5IHRoZSBzYW1lCgpgYGB7cn0KIyMgQ29kZSBmcm9tIFNldGggRmFsY29uClAgPC0gZnVuY3Rpb24oc2l6ZSl7CiAgbkZvdW5kIDwtIDEwCiAgbkRyYXduIDwtIDQwMAogIG5BdENhdCA8LSA0MAogIG5Ob3RBdENhdCA8LSBzaXplIC0gbkF0Q2F0CiAgcGh5cGVyKG5Gb3VuZC0xLCBuQXRDYXQsbk5vdEF0Q2F0LG5EcmF3bixsb3dlci50YWlsID0gRkFMU0UpCn0KUCgxMDAwKQpQKDUwMDApCmBgYAoKCgojIyBQcmVwYXJpbmcgdGhlIGRhdGEgZm9yIGFuIG92ZXItcmVwcmVzZW50YXRpb24gdGVzdAoKQXMgYmVmb3JlLCB3ZSBkb3dubG9hZCB0aGUgZGF0YSBmcm9tIEdFTy4KCmBgYHtyIGNhY2hlPVRSVUV9CmxpYnJhcnkoR0VPcXVlcnkpCmxpYnJhcnkoZ2VuZWZpbHRlcikKdXJsIDwtICJmdHA6Ly9mdHAubmNiaS5uaWguZ292L3B1Yi9nZW8vREFUQS9TZXJpZXNNYXRyaXgvR1NFMzMxMjYvR1NFMzMxMjZfc2VyaWVzX21hdHJpeC50eHQuZ3oiCmZpbGVubSA8LSAiZGF0YS9HU0UzMzEyNl9zZXJpZXNfbWF0cml4LnR4dC5neiIKaWYoIWZpbGUuZXhpc3RzKCJkYXRhL0dTRTMzMTI2X3Nlcmllc19tYXRyaXgudHh0Lmd6IikpIGRvd25sb2FkLmZpbGUodXJsLCBkZXN0ZmlsZT1maWxlbm0pCmNvbG9uRGF0YSA8LSBnZXRHRU8oZmlsZW5hbWU9ZmlsZW5tKQpjb2xvbkRhdGEKYGBgCgpSZW1lbWJlciB0aGF0IHdlIGFsc28gbmVlZCB0byBkbyB0aGUgbG9nJF8yJCB0cmFuc2Zvcm1hdGlvbgoKYGBge3J9CmV4cHJzKGNvbG9uRGF0YSkgPC0gbG9nMihleHBycyhjb2xvbkRhdGEpKQpgYGAKCgpXZSBhcmUgbm93IGdvaW5nIHRvIGNyZWF0ZSBhIGdlbmUgdW5pdmVyc2UgYnkgcmVtb3ZpbmcgZ2VuZXMgZm9yIHdpbGwgbm90IGNvbnRyaWJ1dGUgdG8gdGhlCnN1YnNlcXVlbnQgYW5hbHlzaXMuIFN1Y2ggZmlsdGVyaW5nIGlzIGRvbmUgd2l0aG91dCByZWdhcmRpbmcgdGhlIHBoZW5vdHlwZSB2YXJpYWJsZXMgLSBoZW5jZQphIOKAnW5vbi1zcGVjaWZpY+KAnSBmaWx0ZXIuIEFuIElsbHVtaW5hIEh1bWFuNiBjaGlwIGNvbnRhaW5zIGFyb3VuZCA0OCwwMCBwcm9iZXMsIGJ1dCBsZXNzIHRoYW4KaGFsZiBvZiB0aGVzZSBoYXZlIGVub3VnaCBkZXRhaWxlZCBpbmZvcm1hdGlvbiB0byB1c2VmdWwgZm9yIGEgR08gYW5hbHlzaXMuIFRoZXJlZm9yZSB3ZSByZXN0cmljdCB0aGUgZGF0YXNldCB0byBvbmx5IHByb2JlcyBmb3Igd2hpY2ggd2UgaGF2ZSBhIEVudHJleiBJRC4gTW9yZW92ZXIsIHdlIHdvdWxkIGxpa2Ugb25seSB0aGUgbW9zdC12YXJpYWJsZSBwcm9iZSBmb3IgYSBnaXZlbiBnZW5lLgoKLSBUaGlzIGNhbiBiZSBwZXJmb3JtZWQgYXV0b21hdGljYWxseSB1c2luZyB0aGUgYG5zRmlsdGVyYCBmdW5jdGlvbgotIEZpcnN0IHdlIGhhdmUgdG8gdGVsbCB0aGUgZnVuY3Rpb24gd2UgYXJlIGRlYWxpbmcgd2l0aCBgaWxsdW1pbmFIdW1hbnYzYCBkYXRhCi0gVGhlIG91dHB1dCBvZiBgbnNGaWx0ZXJgIGNvbnRhaW5zIHNldmVyYWwgY29tcG9uZW50cwogICAgKyB3ZSB3YW50IGp1c3QgdGhlIGV4cHJlc3Npb24gc2V0IG9mIGZpbHRlcmVkIHJvd3MsIHNvIHdlIG5lZWQgdG8gc2VsZWN0IHRoaXMKICAgIApgYGB7cn0KbGlicmFyeShnZW5lZmlsdGVyKQphbm5vdGF0aW9uKGNvbG9uRGF0YSkgPC0gImlsbHVtaW5hSHVtYW52MyIKZmlsdCA8LSBuc0ZpbHRlcihjb2xvbkRhdGEpCmNvbG9uRGF0YS5maWx0IDwtIGZpbHQkZXNldAoKYGBgCgoKV2UnbGwgd3JpdGUtb3V0IHRoZSBuYW1lcyBvZiB0aGUgRW50cmV6IGdlbmVzIHRoYXQgd2UgaGF2ZSBzbyB3ZSBjYW4gdXNlIHRoaXMgYXMgb3VyICpHZW5lIFVuaXZlcnNlKi4KCmBgYHtyfQphbm5vLmZpbmFsIDwtIGZEYXRhKGNvbG9uRGF0YS5maWx0KQp1bml2ZXJzZUlkcyA8LSBhbm5vLmZpbmFsJEVudHJlel9HZW5lX0lECmxlbmd0aCh1bml2ZXJzZUlkcykKd3JpdGUudGFibGUodW5pdmVyc2VJZHMsIGZpbGU9ImdlbmVVbml2ZXJzZS50eHQiLHNlcD0iXHQiLHJvdy5uYW1lcyA9IEZBTFNFLHF1b3RlPUZBTFNFKQpgYGAKCldlJ3ZlIHByZXZpb3VzbHkgZXhwbGFpbmVkIHRoZSBmb2xsb3dpbmcgc3RlcHMgdG8gcHJvZHVjZSBhIHRhYmxlIG9mIERFIGdlbmVzLiBIb3dldmVyLCBmb3IgdGhpcyBhbmFseXNpcyB3ZSBhcmUgaW50ZXJlc3RlZCBpbiBhIHJlbGF0aXZlbHkgbG9uZyBnZW5lLWxpc3QgYW5kIGRvIG5vdCBuZWVkIHRvIGJlIHNvIHN0cmluZ2VudCwgc28gd2UgY2FuIGFwcGx5IGEgc3RhbmRhcmQgdC10ZXN0CgotIFRoZSBgcm93dHRlc3RzYCBmdW5jdGlvbiBpcyBhbm90aGVyIHVzZWZ1bCBmdW5jdGlvbiBmb3IgYXBwbHlpbmcgdGhlIHNhbWUgb3BlcmF0aW9uIHRvIHRoZSByb3dzIG9mIGEgbWF0cml4LCBpbiB0aGlzIGNhc2UgYSB0LXRlc3QKCmBgYHtyfQpsaWJyYXJ5KGdlbmVmaWx0ZXIpClNhbXBsZUdyb3VwIDwtIHBEYXRhKGNvbG9uRGF0YSkkc291cmNlX25hbWVfY2gxClBhdGllbnQgPC0gcERhdGEoY29sb25EYXRhKSRjaGFyYWN0ZXJpc3RpY3NfY2gxLjEKCnR0ZXN0cyA8LSByb3d0dGVzdHMoY29sb25EYXRhLmZpbHQsIGZhYz1TYW1wbGVHcm91cCkKY3V0b2ZmIDwtIDAuMDEKCnNlbGVjdGVkIDwtIHR0ZXN0cyRwLnZhbHVlIDwgY3V0b2ZmCnNlbGVjdGVkRW50cmV6SWRzIDwtIGZEYXRhKGNvbG9uRGF0YS5maWx0KSRFbnRyZXpfR2VuZV9JRFtzZWxlY3RlZF0KbGVuZ3RoKHNlbGVjdGVkRW50cmV6SWRzKQpgYGAKCgojIyBVc2luZyBvbmxpbmUgdG9vbHMKCkV2ZW4gdGhvdWdoIHRoaXMgaXMgYSBjb3Vyc2UgYWR2b2NhdGluZyB0aGUgdXNlIG9mIFIgYW5kIEJpb2NvbmR1Y3RvciwgdGhlcmUgYXJlIGluIGZhY3QgbWFueSBvbmxpbmUgdG9vbHMgdGhhdCBjYW4gYXNzaXN0IHdpdGggZ2VuZSBzZXQgYW5hbHlzZXMuIFtEQVZJRF0oaHR0cHM6Ly9kYXZpZC5uY2lmY3JmLmdvdi8pIGlzIGEgdmVyeSBwb3B1bGFyIHRvb2wgYW5kIGlzIGFibGUgdG8gYWNjZXB0IGdlbmUgbGlzdHMgaW4gdGhlIGZvcm0gb2YgRW50cmV6IElEcy4KCldlIHdyaXRlIHRoZSBzZWxlY3RlZCBFbnRyZXogSURzIHRvIGEgdGV4dCBmaWxlLiBTbyB0aGF0IHRoZSBmaWxlIGlzIHJlY29nbmlzZWQgYnkgREFWSUQsIHdlIG5lZWQgdG8gc3BlY2lmeSB0aGUgYXJndW1lbnQgYHJvdy5uYW1lcz1GQUxTRWAgdG8gbWFrZSBzdXJlIHRoYXQgcm93IG51bWJlcnMgZG8gbm90IGFwcGVhciBpbiB0aGUgZmlsZS4KCgpgYGB7cn0Kd3JpdGUudGFibGUoc2VsZWN0ZWRFbnRyZXpJZHMsIGZpbGU9Im15RW50cmV6R2VuZS50eHQiLHNlcD0iXHQiLHF1b3RlPUZBTFNFLHJvdy5uYW1lcyA9IEZBTFNFKQpgYGAKCllvdSBjYW4gdXBsb2FkIHRoaXMgdGV4dCBmaWxlIHRvIHRoZSBEQVZJRCB3ZWJzaXRlLCBhbG9uZyB3aXRoIHRoZSBnZW5lIHVuaXZlcnNlIHdlIGNyZWF0ZWQgcHJldmlvdXNseSwgYW5kIGZvbGxvdyB0aGUgaW5zdHJ1Y3Rpb25zLgoKIyMgT3Zlci1yZXByZXNlbnRhdGlvbiBhbmFseXNpcyBpbiBCaW9jb25kdWN0b3I6IEdPc3RhdHMKCgpUaGUgaHlwZXJHVGVzdCBmdW5jdGlvbiBpcyB1c2VkIHRvIGRvIHRoZSBoeXBlcmdlb21ldHJpYyB0ZXN0IGZvciBHTyB0ZXJtcy4gUmF0aGVyIHRoYW4KcGFzc2luZyBhIGxvbmcgbGlzdCBvZiBwYXJhbWV0ZXJzIHRvIHRoZSBmdW5jdGlvbi4gQW4gb2JqZWN0IG9mIHR5cGUgYEdPSHlwZXJHUGFyYW1zYCBpcwpjcmVhdGVkIHRvIGhvbGQgYWxsIHRoZSBwYXJhbWV0ZXJzIHdlIG5lZWQgdG8gcnVuIHRoZSBoeXBlcmdlb21ldHJpYyB0ZXN0LiBUaGlzIG9iamVjdCBjYW4KdGhlbiBiZSBwYXNzZWQgdG8gYGh5cGVyR1Rlc3RgIG11bHRpcGxlIHRpbWVzIHdpdGhvdXQgaGF2aW5nIHRvIHJlLXR5cGUgdGhlIHBhcmFtZXRlcnMgZWFjaAp0aW1lLiBUaGUgbWVhbmluZ3Mgb2YgdGhlc2UgcGFyYW1ldGVycyBhcmUgYXMgZm9sbG93czoKCiAgLSBgZ2VuZUlkc2AgLSBUaGUgbGlzdCBvZiBpZGVudGlmaWVycyBmb3IgdGhlIGdlbmVzIHRoYXQgd2UgaGF2ZSBzZWxlY3RlZCBhcyBpbnRlcmVzdGluZwogIC0gYHVuaXZlcnNlR2VuZUlkc2AgLSBUaGUgbGlzdCBvZiBpZGVudGlmaWVycyByZXN1bHRpbmcgZnJvbSBub24tc3BlY2lmaWMgZmlsdGVyaW5nCiAgLSBgYW5ub3RhdGlvbmAgLSBUaGUgbmFtZSBvZiB0aGUgYW5ub3RhdGlvbiBwYWNrYWdlIHRoYXQgd2lsbCBiZSB1c2VkCiAgLSBgb250b2xvZ3lgIC0gVGhlIG5hbWUgb2YgdGhlIEdPIG9udG9sb2d5IHRoYXQgd2lsbCBiZSB0ZXN0ZWQ7IGVpdGhlciBCUCwgQ0Mgb3IgTUYKICAtIGBwdmFsdWVjdXRvZmZgIC0gcC12YWx1ZSB0aGF0IHdlIHdpbGwgdXNlIHRvIHNlbGVjdCBzaWduaWZpY2FudCBHTyB0ZXJtcwogIC0gYHRlc3REaXJlY3Rpb25gIC0gRWl0aGVyIOKAnW92ZXLigJ0gb3Ig4oCddW5kZXLigJ0gZm9yIG92ZXIgb3IgdW5kZXIgcmVwcmVzZW50ZWQgdGVybXMgcmVzcGVjdGl2ZWx5CiAgLSBgY29uZGl0aW9uYWxgIC0gQSBtb3JlIHNvcGhpc3RpY2F0ZWQgZm9ybSBvZiBoeXBlcmdlb21ldHJpYyB0ZXN0LCB3aGljaCB0YWtlcyB0aGUgcmVsYXRpb25zaGlwcyBiZXR3ZWVuIHRlcm1zIGluIHRoZSBHTyBncmFwaCBjYW4gYmUgdXNlZCBpZiB0aGlzIGlzIHNldCB0byBUUlVFLiBGb3IgdGhpcyBwcmFjdGljYWwgd2Ugd2lsbCBrZWVwIGNvbmRpdGlvbmFsID0gRkFMU0UKCiAgCmBgYHtyIGNhY2hlPVRSVUV9CmxpYnJhcnkoR09zdGF0cykKcGFyYW1zID0gbmV3ICgiR09IeXBlckdQYXJhbXMiICwgZ2VuZUlkcyA9IHNlbGVjdGVkRW50cmV6SWRzICwgCiAgICAgICAgICAgICAgdW5pdmVyc2VHZW5lSWRzID0gdW5pdmVyc2VJZHMgLCBhbm5vdGF0aW9uID0gImlsbHVtaW5hSHVtYW52MyIgLAogICAgICAgICAgICAgIG9udG9sb2d5ID0gICJCUCIgLCBwdmFsdWVDdXRvZmYgPSAwLjA1ICwgY29uZGl0aW9uYWwgPSBGQUxTRSAsCiAgICAgICAgICAgICAgdGVzdERpcmVjdGlvbiA9ICJvdmVyIikKaGdPdmVyID0gaHlwZXJHVGVzdChwYXJhbXMpCmhnT3ZlcgpgYGAKClRoZSBgc3VtbWFyeWAgZnVuY3Rpb24gY2FuIGJlIHVzZWQgdG8gdmlldyB0aGUgcmVzdWx0cyBvZiB0aGUgdGVzdCBpbiBtYXRyaXggZm9ybS4gVGhlIHJvd3MKb2YgdGhlIG1hdHJpeCBhcmUgYXJyYW5nZWQgaW4gb3JkZXIgb2Ygc2lnbmlmaWNhbmNlLiBUaGUgcC12YWx1ZSBpcyBzaG93biBmb3IgZWFjaCBHTyB0ZXJtCmFsb25nIHdpdGggd2l0aCB0b3RhbCBudW1iZXIgb2YgZ2VuZXMgZm9yIHRoYXQgR08gdGVybSwgbnVtYmVyIG9mIGdlbmVzIHdlIHdvdWxkIGJlIGV4LQpwZWN0IHRvIGFwcGVhciBpbiB0aGUgZ2VuZSBsaXN0IGJ5IGNoYW5jZSBhbmQgdGhhdCBudW1iZXIgdGhhdCB3ZXJlIG9ic2VydmVkLiBBIGRlc2NyaXB0aXZlCm5hbWUgaXMgYWxzbyBnaXZlbiBmb3IgZWFjaCB0ZXJtLiBUaGUgcmVzdWx0cyBjYW4gYWxzbyBiZSBwcmludGVkIG91dCB0byBhIEhUTUwgcmVwb3J0IHVzaW5nIHRoZSBgaHRtbFJlcG9ydGAgZnVuY3Rpb24uCgpgYGB7ciBjYWNoZT1UUlVFfQpzdW1tYXJ5IChoZ092ZXIpWzE6MjAgLCBdCmh0bWxSZXBvcnQoaGdPdmVyLGZpbGU9ImdvLXN1bW1hcnkuaHRtbCIpCgoKYGBgCgoKCiMjIFRoZW9yeSBQYXJ0IElJOiBUaHJlc2hvbGQtZnJlZQoKRm9yIHRoZXNlIHRlc3RzLCB3ZSBkb24ndCBoYXZlIHRvIHNwZWNpZnkgYSBzdGF0aXN0aWNhbCB0aHJlc2hvbGQgYW5kIHVzZSB0aGUgdGVzdCBzdGF0aXN0aWNzIGZyb20gKmFsbCogZ2VuZXMgYXMgdGhlIGlucHV0IHRvIHRoZSB0ZXN0LiBUaGUgcG9wdWxhciAqR2VuZSBTZXQgRW5yaWNobWVudCBBbmFseXNpcyAoR1NFQSkqIHVzZXMgdGhpcyBhcHByb2FjaC4gVGhlc2UgdGVzdHMgY2FuIGJlIHVzZWQgdG8gZGV0ZWN0IGRpZmZlcmVudGlhbCBleHByZXNzaW9uIGZvciBhIGdyb3VwIG9mIGdlbmVzLCBldmVuIHdoZW4gdGhlIGVmZmVjdHMgYXJlIHRvbyBzbWFsbCBvciB0aGVyZSBpcyB0b28gbGl0dGxlIGRhdGEgdG8gZGV0ZWN0IHRoZSBnZW5lcyBpbmRpdmlkdWFsbHkuCgohW2dzZWFdKGltYWdlcy9HU0VBLnBuZykKCipTdWJyYW1hbmlhbiBldCBhbCwgUE5BUyAyMDA1KgoKVGhlIEJyb2FkIGluc3RpdHV0ZSBwcm92aWRlcyBbYSB2ZXJzaW9uIG9mIEdTRUFdKGh0dHA6Ly9zb2Z0d2FyZS5icm9hZGluc3RpdHV0ZS5vcmcvZ3NlYS9pbmRleC5qc3ApIHRoYXQgY2FuIGJlIHJ1biB2aWEgYSBqYXZhIGFwcGxpY2F0aW9uLiBIb3dldmVyLCBpdCByZXF1aXJlcyB5b3UgdG8gY29udmVydCB5b3VyIGRhdGEgaW50byBhIHBhcnRpY3VsYXIgZm9ybWF0LiBBIG1vcmUgY29udmVuaWVudCBvcHRpb24gaXMgWyoqKkdlbmVUcmFpbCoqKl0oaHR0cDovL2dlbmV0cmFpbC5iaW9pbmYudW5pLXNiLmRlL2VucmljaG1lbnRfYW5hbHlzaXMucGhwP2pzPTEmY2M9MSkgd2hpY2ggaGFzIHRoZSBvcHRpb24gdG8gZG8gYSBHU0VBLXN0eWxlIGFuYWx5c2lzLgoKVGhlIHJlcXVpcmVkIGlucHV0IHRleHQgY29udGFpbnMgYSBzZXQgb2YgZ2VuZSBpZGVudGlmaWVycyB0aGF0IGFyZSBvcmRlcmVkIGFjY29yZGluZyB0byBzb21lIHRlc3Qgc3RhdGlzdGljLiBUbyBwcm9kdWNlIHN1Y2ggYSBmaWxlLCB3ZSBmaXJzdCBoYXZlIHRvIG9yZGVyIHRoZSB0LXN0YXRpc3RpY3MgZnJvbSBvdXIgdGVzdCBgdHRlc3RzJHN0YXRpc3RpY2Agd2l0aCB0aGUgYG9yZGVyYCBmdW5jdGlvbi4KCmBgYHtyfQpnc2VhTGlzdCA8LSBhbm5vLmZpbmFsJEVudHJlel9HZW5lX0lEW29yZGVyKHR0ZXN0cyRzdGF0aXN0aWMsZGVjcmVhc2luZyA9IFRSVUUpXQp3cml0ZS50YWJsZShnc2VhTGlzdCwgZmlsZT0iZ2VuZXRyYWlsLWdzZWEudHh0IixzZXA9Ilx0Iixyb3cubmFtZXMgPSBGQUxTRSxxdW90ZT1GQUxTRSkKYGBgCgpOb3RlIHRoYXQgR2VuZXRyYWlsIGNhbiBhbHNvIGJlIHVzZWQgZm9yIHRoZSBPdmVyLVJlcHJlc2VudGF0aW9uIGFuYWx5c2lzIHRoYXQgd2Ugc2F3IGluIHRoZSBwcmV2aW91cyBzZWN0aW9uLgoKIyMgQSBUaHJlc2hvbGQtZnJlZSB0ZXN0IGluIEJpb2NvbmR1Y3RvcgoKQSBzaW1wbGUgdmVyc2lvbiBvZiB0aGlzIHRlc3QgaXMgaW1wbGVtZW50ZWQgaW4gdGhlIGBnZW5lU2V0VGVzdGAgZnVuY3Rpb24gaW4gYGxpbW1hYCwKd2hpY2ggY29tcHV0ZXMgYSBwLXZhbHVlIHRvIHRlc3QgdGhlIGh5cG90aGVzaXMgdGhhdCB0aGUgc2VsZWN0ZWQgZ2VuZXMgaGF2ZSBtb3JlIGV4dHJlbWUKdGVzdC1zdGF0aXN0aWNzIHRoYW4gb25lIG1pZ2h0IGV4cGVjdCBieSBjaGFuY2UuIE1vcmVvdmVyLCBzZXBhcmF0ZSB0ZXN0cyBjYW4gYmUgcGVyZm9ybWVkCnRvIHNlZSBpZiB0aGUgc2VsZWN0ZWQgZ2VuZXMgYXJlIHVwLXJlZ3VsYXRlZCAoYGFsdGVybmF0aXZlPXVwYCkgb3IgZG93bi1yZWd1bGF0ZWQgKGBhbHRlcm5hdGl2ZT1kb3duYCkuIFRoZSBkZWZhdWx0IGlzIHRvIHRlc3QgZm9yIGV4dHJlbWUgc3RhdGlzdGljcyByZWdhcmRsZXNzIG9mIHNpZ24uCgpUbyBkZW1vbnN0cmF0ZSwgd2Ugd2lsbCBwaWNrIDUwIGdlbmVzIGF0IHJhbmRvbS4gV2UgaGF2ZSB0byBzdXBwbHkgdGhlIGVudGlyZSBzZXQgb2YgdGVzdCBzdGF0aXN0aWNzIGFsb25nIGluZGljZXMgb2YgaW50ZXJlc3QuIFRoZSByZXN1bHQgb2YgdGhlIGBiYXJjb2RlYCBwbG90IGJlbG93IHNob3dzIHRoZXJlIGlzIG5vIHBhcnRpY3VsYXIgdHJlbmQgZm9yIGdlbmVzIHdlIGhhdmUgcGlja2VkIGJlIHVwLSBvciBkb3duLXJlZ3VsYXRlZC4KCi0gSGVyZSBgc2FtcGxlYCBwaWNrcyBhIHJhbmRvbSBzZXQgb2YgYG5gIHZhbHVlcyBmcm9tIGEgZ2l2ZW4gdmVjdG9yCi0gQXMgd2UgZXhwZWN0LCB0aGUgcmFuZG9tIHJvd3MgYXJlIG5laXRoZXIgc2tld2VkIHRvd2FyZHMgdG8gdG9wIG9yIGJvdHRvbSBvZiB0aGUgcmFua2VkIGxpc3QKCmBgYHtyfQpsaWJyYXJ5IChsaW1tYSkKCnJhbmRHZW5lcyA8LSBzYW1wbGUoMTpucm93KHR0ZXN0cyksNTApCnJhbmRHZW5lcwoKZ2VuZVNldFRlc3QgKGluZGV4ID0gcmFuZEdlbmVzICwgc3RhdGlzdGljcyA9IHR0ZXN0cyRzdGF0aXN0aWMpCgpnZW5lU2V0VGVzdCAoaW5kZXggPSByYW5kR2VuZXMgLCBzdGF0aXN0aWNzID0gdHRlc3RzJHN0YXRpc3RpYyAsCiAgYWx0ZXJuYXRpdmUgPSAiZG93biIgKQoKZ2VuZVNldFRlc3QgKGluZGV4ID0gcmFuZEdlbmVzICwgc3RhdGlzdGljcyA9IHR0ZXN0cyRzdGF0aXN0aWMgLAogIGFsdGVybmF0aXZlID0gInVwIiApCgoKYGBgCgpBIHVzZWZ1bCB3YXkgb2YgdmlzdWFsaXNpbmcgdGhlIHJlc3VsdHMgaXMgd2l0aCBhIGBiYXJjb2RlcGxvdGAuIFRoZSBzdGF0aXN0aWNzIGFyZSByYW5rZWQgbGVmdCB0byByaWdodCBmcm9tIGxhcmdlc3QgdG8gc21hbGxlc3QuIFRoZSByYW5rZWQgc3RhdGlzdGljcyBhcmUgcmVwcmVzZW50ZWQgYnkgYSBzaGFkZWQgYmFyIG9yIGJlZCwgYW5kIHRoZSBwb3NpdGlvbnMgb2YgdGhlIHNwZWNpZmllZCBzdWJzZXRzIGFyZSBtYXJrZWQgYnkgdmVydGljYWwgYmFycywgZm9ybWluZyBhIHBhdHRlcm4gbGlrZSBhIGJhcmNvZGUuIEluc3BpcmVkIGJ5IEdTRUEsIHRoZSBsaW5lIGdyYXBoIGFib3ZlIHRoZSBwbG90IChhbiAqZW5yaWNobWVudCB3b3JtKikgZ2l2ZXMgYSBzZW5zZSBvZiB3aGVyZWFib3V0cyB0aGUgZW5yaWNobWVudCBpcyBoaWdoZXN0LiAKYGBge3J9CmJhcmNvZGVwbG90IChzdGF0aXN0aWNzID0gYXMubnVtZXJpYyh0dGVzdHMkc3RhdGlzdGljKSAsIGluZGV4ID0gcmFuZEdlbmVzICkKYGBgCgoKT24gdGhlIG90aGVyIGhhbmQsIHdlIGNvdWxkIGRlbGliZXJhdGVseSBwaWNrIGdlbmVzIHdpdGggYSBzaWduaWZpY2FudCBwLXZhbHVlIGFuZCBub3RpY2UgaG93IHRoZSByZXN1bHRzIGFyZSBhbHRlcmVkLgoKCmBgYHtyfQoKbXlHZW5lcyA8LSB3aGljaCh0dGVzdHMkcC52YWx1ZSA8IDAuMDEpWzE6NTBdCgpnZW5lU2V0VGVzdCAoaW5kZXggPSBteUdlbmVzICwgc3RhdGlzdGljcyA9IHR0ZXN0cyRzdGF0aXN0aWMpCgpnZW5lU2V0VGVzdCAoaW5kZXggPSBteUdlbmVzICwgc3RhdGlzdGljcyA9IHR0ZXN0cyRzdGF0aXN0aWMgLAogIGFsdGVybmF0aXZlID0gImRvd24iICkKCmdlbmVTZXRUZXN0IChpbmRleCA9IG15R2VuZXMgLCBzdGF0aXN0aWNzID0gdHRlc3RzJHN0YXRpc3RpYyAsCiAgYWx0ZXJuYXRpdmUgPSAidXAiICkKCmJhcmNvZGVwbG90IChzdGF0aXN0aWNzID0gYXMubnVtZXJpYyh0dGVzdHMkc3RhdGlzdGljKSAsIGluZGV4ID0gbXlHZW5lcyApCgpgYGAKCklmIHdlIGhhdmUgYSBzcGVjaWZpYyBxdWVyeSBhYm91dCBhIHNldCBvZiBnZW5lcywgd2UgbmVlZCB0byBsb2NhdGUgdGhlbSBpbiB0aGUgbGlzdCBvZiByYW5rZWQgc3RhdGlzdGljcy4gCgotIFdlIGhhdmUgYWxyZWFkeSBzZWVuIG9mIGV4YW1wbGUgb2Ygc2VhcmNoaW5nIGZvciBwYXJ0aWN1bGFyIGdlbmUgc3ltYm9scyBpbiBvdXIgb3V0cHV0IHRhYmxlcy4KLSBXZSB3ZXJlIGNhcmVmdWwgdG8gbWFrZSBzdXJlIHRoZSBhbm5vdGF0aW9uIHdhcyBpbiB0aGUgc2FtZSBvcmRlciBhcyB0aGUgZXhwcmVzc2lvbiBtYXRyaXgsIHNvIHRoaXMgc2hvdWxkIGJlIHN0cmFpZ2h0Zm9yd2FyZAoKYGBge3J9Cm15bGlzdCA8LSBjKCJMT0M0NDEwNjYiLCJBUkYzIiwiRk1OTDMiLCJDU0YxUiIsIlhMS0QxIiwiVFRSQVAiLCJETVdEIiwiU1lOUE8yTCIsIlBJTFJCIiwiTEFNUDMiKQpteUdlbmVzIDwtIHdoaWNoKGFubm8uZmluYWwkU3ltYm9sICVpbiUgbXlsaXN0KQpnZW5lU2V0VGVzdCAoaW5kZXggPSBteUdlbmVzICwgc3RhdGlzdGljcyA9IHR0ZXN0cyRzdGF0aXN0aWMpCgpnZW5lU2V0VGVzdCAoaW5kZXggPSBteUdlbmVzICwgc3RhdGlzdGljcyA9IHR0ZXN0cyRzdGF0aXN0aWMgLAogIGFsdGVybmF0aXZlID0gImRvd24iICkKCmdlbmVTZXRUZXN0IChpbmRleCA9IG15R2VuZXMgLCBzdGF0aXN0aWNzID0gdHRlc3RzJHN0YXRpc3RpYyAsCiAgYWx0ZXJuYXRpdmUgPSAidXAiICkKCmJhcmNvZGVwbG90IChzdGF0aXN0aWNzID0gYXMubnVtZXJpYyh0dGVzdHMkc3RhdGlzdGljKSAsIGluZGV4ID0gbXlHZW5lcykKYGBgCgpIb3dldmVyLCBpZiB3ZSB3YW50IHRvIGludGVycm9nYXRlIHRoZSByZXN1bHRzIGZvciBhIHBhcnRpY3VsYXIgR08gb3IgS0VHRyBwYXRod2F5IHdlIGNvdWxkIGxvb2sgdXAgdGhlIGlkZW50aWZpZXJzIG1hbnVhbGx5LCBvciB1c2Ugc29tZSB1c2VmdWwgZnVuY3Rpb25hbGl0eSBpbiBCaW9jb25kdWN0b3IuCgojIEFubm90YXRpb24KCiMjIENvbnZlcnRpbmcgYmV0d2VlbiBkaWZmZXJlbnQgaWRlbnRpZmllcnMKClRoZXJlIGFyZSBmYWNpbGl0aWVzIGluIEJpb2NvbmR1Y3RvciBmb3IgY29udmVydGluZyBiZXR3ZWVuIGRpZmZlcmVudCBpZGVudGlmaWVyIChlLmcuIEVudHJleiwgRW5zZW1ibCwgVW5pZ2VuZSkgc2NoZW1lcy4gRm9yIGVhY2ggb3JnYW5pc20sIHRoZXJlIGlzIGEgcHJlLWJ1aWx0ICpkYXRhYmFzZSogcGFja2FnZSB0aGF0IGFsbG93cyB2YXJpb3VzIHF1ZXJpZXMgdG8gYmUgbWFkZS4gSW4gdGhpcyBleGFtcGxlLCB3ZSB3aWxsIHVzZSB0aGUgYG9yZy5Icy5lZy5kYmAgcGFja2FnZSBmb3IgaHVtYW5zLiBPdGhlciBvcmdhbmlzbXMgY2FuIGJlIGZvdW5kIG9uIHRoZSBsaXN0IG9mIFtCaW9jb25kdWN0b3IgYW5ub3RhdGlvbiBwYWNrYWdlc10oaHR0cDovL2Jpb2NvbmR1Y3Rvci5vcmcvcGFja2FnZXMvcmVsZWFzZS9CaW9jVmlld3MuaHRtbCNfX19Bbm5vdGF0aW9uRGF0YSk7IGp1c3QgbG9vayBmb3IgcGFja2FnZXMgdGhhdCBzdGFydCAqb3JnLiouCgpBbiBhZHZhbnRhZ2Ugb2YgdXNpbmcgdGhlc2UgcGFja2FnZXMsIHJhdGhlciB0aGFuIHdlYiBzZXJ2aWNlcyBzdWNoIGFzIGJpb21hcnQsIGlzIHRoYXQgb2ZmbGluZSBxdWVyaWVzIGFyZSBwb3NzaWJsZS4KCkVhY2ggb3JnYW5pc20gaGFzIGEgc2VyaWVzIG9mICpjb2x1bW5zKiBhbmQgKmtleXMqIGRlZmluZWQuIFRoZSAqa2V5cyogYXJlIHRoZSBpZGVudGlmZXJzIHRoYXQgeW91IHdhbnQgdG8gcXVlcnksIGFuZCB0aGUgKmNvbHVtbnMqIGRlZmluZSB0aGUgaW5mb3JtYXRpb24gdGhhdCB5b3Ugd2FudCB0byByZXRyaWV2ZS4KCmBgYHtyfQpsaWJyYXJ5KG9yZy5Icy5lZy5kYikKY29sdW1ucyhvcmcuSHMuZWcuZGIpCmtleXR5cGVzKG9yZy5Icy5lZy5kYikKYGBgCgpMZXRzJyBmaW5kIG91dCB3aGF0IHRoZSBFbnRyZXogSUQncyBhcmUgZm9yIHRoZSBnZW5lcyAqVFA1MyosICpCUkNBMSogYW5kICpQVEVOKi4gV2Ugc3BlY2lmeSBvdXIgYGtleXNgIGFzIHRoZSB2ZWN0b3Igb2YgdGhlc2UgbmFtZXMsIG91ciBga2V5dHlwZWAgaXMgYFNZTUJPTGAgKHdoaWNoIG11c3QgYmUgaW4gdGhlIHJlc3VsdCB3ZSBnZXQgYmFjayBmcm9tIGRvaW5nIGBrZXl0eXBlc2ApIGFuZCB0aGUgYGNvbHVtbnNgIGlzIGBFTlRSRVpJRGAuIAoKYGBge3J9CnNlbGVjdChvcmcuSHMuZWcuZGIsIGtleXM9YygiVFA1MyIsIkJSQ0ExIiwiUFRFTiIpLCBrZXl0eXBlID0gIlNZTUJPTCIsY29sdW1ucyA9ICJFTlRSRVpJRCIpCmBgYAoKV2UgY2FuIGNob29zZSB0byByZXR1cm4gbXVsdGlwbGUgYml0cyBvZiBpbmZvcm1hdGlvbiBieSBjaGFuZ2luZyB0aGUgYGNvbHVtbnNgIGFyZ3VtZW50LiBJbiBzb21lIGNhc2VzIHRob3VnaCwgeW91IGNvdWxkIGdldCBtYW55IHJlc3VsdHMgcmV0dXJuZWQgZm9yIHRoZSBzYW1lIGtleS4KCmBgYHtyfQpzZWxlY3Qob3JnLkhzLmVnLmRiLCBrZXlzPWMoIlRQNTMiLCJCUkNBMSIsIlBURU4iKSwga2V5dHlwZSA9ICJTWU1CT0wiLGNvbHVtbnMgPSBjKCJFTlRSRVpJRCIsIkVOU0VNQkwiLCJVTklQUk9UIikpCmBgYAoKV2UgY2FuIGZpbmQgb3V0IHdoaWNoIGdlbmVzIGJlbG9uZyB0byBhIHBhcnRpY3VsYXIgcGF0aC4gZS5nLiB0aGUgKkNlbGwgQ3ljbGUqIHBhdGh3YXkgaXMgS0VHRzogMDQxMTAgYW5kIHRoZSBga2V5dHlwZWAgaXMgYFBBVEhgLgoKYGBge3J9CnNlbGVjdChvcmcuSHMuZWcuZGIsIGtleXM9IjA0MTEwIiwga2V5dHlwZSA9ICJQQVRIIixjb2x1bW5zPWMoIlNZTUJPTCIsIkVOVFJFWklEIikpCmBgYAoKR2VuZS1PbnRvbG9neSB0ZXJtcyBjYW4gYmUgcXVlcmllZCB1c2luZyB0aGUgYEdPQUxMYCBrZXl0eXBlLiBBbHNvIHJldHVybmVkIGluIHRoZSByZXN1bHQgYXJlIHRoZSBHTyAqZXZpZGVuY2UgY29kZXMqIGFuZCBvbnRvbG9neSB0eXBlIChCUCA9IEJpb2xvZ2ljYWwgUHJvY2VzcywgTUYgPSBNb2xlY3VsYXIgRnVuY3Rpb24pLgoKRm9yIGV4YW1wbGUsIHdlIG1pZ2h0IGJlIGludGVyZXN0ZWQgaW4gdGhlIEdPIHRlcm0gKkdPOjAwMTYwNzIqIGFuZCBiZSBpbnRlcmVzdGVkIGluIHdoZXRoZXIgdGhlc2UgZ2VuZXMgYXJlIHVwLSBvciBkb3duLXJlZ3VsYXRlZCBpbiBvdXIgc3R1ZHkuIFRoZSBmaXJzdCBzdGVwIHdvdWxkIGJlIHRvIHJldHJpZXZlIHRoZSBFbnRyZXogSURzIHRoYXQgY29ycmVzcG9uZCB0byB0aGlzIHRlcm0uCgoKYGBge3J9CmdvUXVlcnkgPC0gc2VsZWN0KG9yZy5Icy5lZy5kYiwga2V5cz0iR086MDAxNjA3MiIsIGtleXR5cGUgPSAiR09BTEwiLGNvbHVtbnM9YygiU1lNQk9MIiwiRU5UUkVaSUQiKSkKaGVhZChnb1F1ZXJ5KQpgYGAKCldlIGNhbiBtYXRjaCB0aGVzZSBFbnRyZXogSURzIHRvIG91ciBhbm5vdGF0aW9uIG9iamVjdCBhbmQgcHJvY2VlZCBhcyBiZWZvcmUuCgpgYGB7cn0KbXlHZW5lcyA8LSB3aGljaChhbm5vLmZpbmFsJEVudHJleiAlaW4lIGdvUXVlcnkkRU5UUkVaKQpnZW5lU2V0VGVzdCAoaW5kZXggPSBteUdlbmVzICwgc3RhdGlzdGljcyA9IHR0ZXN0cyRzdGF0aXN0aWMpCmJhcmNvZGVwbG90IChzdGF0aXN0aWNzID0gYXMubnVtZXJpYyh0dGVzdHMkc3RhdGlzdGljKSAsIGluZGV4ID0gbXlHZW5lcyApCmBgYAoK