diff --git a/experiments.tex b/experiments.tex
index 3e195c3995d72dfe86f3dbbf91b412c1b940a777..bc9da62ccf303bbac150407ae39f44e24ae39051 100644
--- a/experiments.tex
+++ b/experiments.tex
@@ -38,7 +38,7 @@ For my first experiment, I formulated four prompts with slightly different wordi
As you can see, $\delta_{2}$ is more specific, requesting the style of a fandom article, whereas $\delta_{3}$ is less precise, asking only for an overview without specifying a particular format. The last prompt is similar to $\delta_{1}$ but is intentionally faulty by missing characters.These different prompts are used to determine the overall effects of various prompt wordings and faulty instructions on the language model.
-% this method isn't ideal, as regularly eliminating sentences could omit important context.
+
In this first experiment, I selected additional information from the book by filtering for every sentence in which the character's name occurred at least once. Since the number of tokens might exceed the maximum input size of the LLaMA model, I removed every $n$-th sentence, where $n$ is calculated in such a way that the query size fits perfectly.
Additionally, because characters are more likely to be introduced in the first sentences where they appear in the book, I added an additional cutoff $\alpha$. This cutoff represents the percentage of relevant sentences (with character name occurrences) to which every sentence with name occurence will be taken, so the rule of taking every $n$-th sentence only affects sentences after the cutoff. Overall the passage retrieval for this experiment $R_{base}$ works as follows. Let $S = \{s_i \mid 1 \leq i \leq k \}$ be the set of size $k$ which contains all relevant sentences containig the character and $l$ be the maximum inputsize of the Llama query. We first definde a function $S_{t}(a, b) = \{ s_{ti} \mid a \cdot k \leq ti \leq b \cdot k \}$, that enables a range selection of sentences with a lower and upper limit and a parameter $t$ for the stepsize. If we now choose our $n$ the right way
\[n = \begin{cases}
@@ -707,14 +707,13 @@ Quantization is a method used to decrease the computational and memory demands o
\subsection{Analysis}
+Obviously the method of passage retrieval used for this experiment isn't ideal, as regularly eliminating sentences could omit important context, also at this stage, the process of fetching fandom articles wasn't complete, resulting in a dataset with some duplicates and missing characterizations. Despite these limitations, the data is still sufficient to show two important aspects of the data. First, the results with passage retrieval are at least as good as, or already slightly better than, those without. Second, the results vary only slightly across the four different prompts. Befor we investigate that further lets have an more detailed look at the results.\\
-At this stage, the process of fetching fandom articles wasn't complete, resulting in a dataset with some duplicates and missing characterizations. Despite these limitations, the data is still sufficient to demonstrate two points. First, the results with additional passages are at least as good as, or slightly better than, those without. Second, the results vary slightly across the four different prompts.\\
-
-As we can see, the BLEU scores from the results of each prompt mostly improve after passage retrieval. Although the maximum values of $\delta_{1}$ and $\delta_{2}$ have decreased slightly in $\delta_{1}'$ and $\delta_{2}'$, the minimum values, Q1, and Q3 have significantly increased, as observed in the box plots. For both $\delta_{3}$ and $\delta_{4}$, every box plot quartile has improved.\\
+As we can see, the BLEU scores of each prompt mostly improve after passage retrieval. Although the maximum values of $\delta_{1}$ and $\delta_{2}$ have decreased slightly in $\delta_{1}'$ and $\delta_{2}'$, the minimum values, Q1, and Q3 have significantly increased, as observed in the box plots. For both $\delta_{3}$ and $\delta_{4}$, every box plot quartile has improved.\\
-For BERTScore, the improvement isn't quite as visible. In fact, the upper quartiles have a lower maximum after passage retrieval, but Q1-Q3 has improved slightly for every prompt. Consequently, the results are more compact. Some outliers close to the maximum in Q4 might score so high prior to passage retrieval due to Llama being trained on similar articles to the fandom articles. Especially when generating summaries for main characters, Llama might already have a great knowledge base for that character, and relying solely on the additional passed sentences might therefore be counterproductive.\\
+For BERTScore, the improvement isn't quite as visible. In fact, the upper quartiles have a lower maximum after passage retrieval, but Q1-Q3 has improved slightly for every prompt. Consequently, the results are more compact. Some outliers close to the maximum in Q4 might score so high prior to passage retrieval due to Llama being trained on similar information to the fandom articles. Especially when generating summaries for main characters, Llama might already have a great knowledge base for that character, and relying solely on the additional passed sentences might therefore be hindering in generating a good characterization.\\
-In summary, semantically, the results haven't improved significantly. The sentences passed to the LLM provide too little information about the general character yet.
+In summary, semantically, the results have only improved slightly and the different wordings in the prompts definately have an influence on the results average and variance (ref figure).
diff --git a/masterthesis.aux b/masterthesis.aux
index cbed3c29f117417efd45b50eb60def704ed7bcd6..9a51daf60bb18cbd34f36d14389f0da645cc6acd 100644
--- a/masterthesis.aux
+++ b/masterthesis.aux
@@ -42,7 +42,7 @@
\BKM@entry{id=17,dest={73656374696F6E2E352E31},srcline={6}}{5C3337365C3337375C303030425C303030615C303030735C303030655C3030305C3034305C303030455C303030785C303030705C303030655C303030725C303030695C3030306D5C303030655C3030306E5C30303074}
\BKM@entry{id=18,dest={73756273656374696F6E2E352E312E31},srcline={124}}{5C3337365C3337375C303030525C303030655C303030735C303030755C3030306C5C303030745C30303073}
\BKM@entry{id=19,dest={73756273656374696F6E2E352E312E32},srcline={709}}{5C3337365C3337375C303030415C3030306E5C303030615C3030306C5C303030795C303030735C303030695C30303073}
-\BKM@entry{id=20,dest={73656374696F6E2E352E32},srcline={722}}{5C3337365C3337375C303030535C303030655C3030306C5C303030655C303030635C303030745C303030655C303030645C3030305C3034305C303030455C3030306D5C303030625C303030655C303030645C303030645C303030655C303030645C3030305C3034305C303030635C303030685C303030755C3030306E5C3030306B5C30303073}
+\BKM@entry{id=20,dest={73656374696F6E2E352E32},srcline={721}}{5C3337365C3337375C303030535C303030655C3030306C5C303030655C303030635C303030745C303030655C303030645C3030305C3034305C303030455C3030306D5C303030625C303030655C303030645C303030645C303030655C303030645C3030305C3034305C303030635C303030685C303030755C3030306E5C3030306B5C30303073}
\@input{conclusion.aux}
\BKM@entry{id=21,dest={636861707465722E36},srcline={1}}{5C3337365C3337375C303030435C3030306F5C3030306E5C303030635C3030306C5C303030755C303030735C303030695C3030306F5C3030306E}
\BKM@entry{id=22,dest={73656374696F6E2E362E31},srcline={3}}{5C3337365C3337375C303030735C303030755C3030306D5C3030306D5C303030615C303030725C30303079}
diff --git a/masterthesis.fdb_latexmk b/masterthesis.fdb_latexmk
index 62401a9824666987971c1fd127cfd11b999cb4be..89bc30231bc76cb3329bbb47a16e5ca2fec8389f 100644
--- a/masterthesis.fdb_latexmk
+++ b/masterthesis.fdb_latexmk
@@ -1,12 +1,12 @@
# Fdb version 4
-["biber masterthesis"] 1718729071.27769 "masterthesis.bcf" "masterthesis.bbl" "masterthesis" 1719004774.52743 0
+["biber masterthesis"] 1718729071.27769 "masterthesis.bcf" "masterthesis.bbl" "masterthesis" 1719008558.24882 0
"_bibliography.bib" 1716391331 6346 b914bbdad333d562f7ed4fca08e399d0 ""
- "masterthesis.bcf" 1719004774 108917 e03a5bf58039b649aef246b5adda42f8 "pdflatex"
+ "masterthesis.bcf" 1719008558 108917 e03a5bf58039b649aef246b5adda42f8 "pdflatex"
(generated)
"masterthesis.bbl"
"masterthesis.blg"
(rewritten before read)
-["pdflatex"] 1719004770.22626 "c:/Users/danie/Desktop/Masterarbbeit NLP/masterthesis.tex" "masterthesis.pdf" "masterthesis" 1719004774.52794 0
+["pdflatex"] 1719008553.74917 "c:/Users/danie/Desktop/Masterarbbeit NLP/masterthesis.tex" "masterthesis.pdf" "masterthesis" 1719008558.24929 0
"C:/Users/danie/AppData/Local/MiKTeX/fonts/map/pdftex/pdftex.map" 1718623652 82507 a9eef8696c92627b87e9975edea03b2a ""
"C:/Users/danie/AppData/Local/MiKTeX/miktex/data/le/pdftex/pdflatex.fmt" 1714948752 24236005 173a77b29316c5b8c9fa5d48235cc505 ""
"C:/Users/danie/AppData/Local/Programs/MiKTeX/fonts/enc/dvips/base/8r.enc" 1458473887 4993 80dc9bab7f31fb78a000ccfed0e27cab ""
@@ -289,28 +289,28 @@
"C:/Users/danie/AppData/Local/Programs/MiKTeX/tex/latex/xcolor/xcolor.sty" 1700127522 55487 80a65caedd3722f4c20a14a69e785d8f ""
"_eidversicherung.tex" 1714902046 922 feed3b62108272e791d7f65fdc1d7c84 ""
"_preamble.sty" 1714948947 2576 04bee0dc31ab20447aa0afe0a876cea2 ""
- "_titlepage.aux" 1719004772 4054 5ae16f9b27d4dbdf1829efb7bed43b88 "pdflatex"
+ "_titlepage.aux" 1719008555 4054 5ae16f9b27d4dbdf1829efb7bed43b88 "pdflatex"
"_titlepage.tex" 1716985422 1384 1962e7b62f97fe2185bbcd860e30e2b5 ""
- "appendix.aux" 1719004774 4273 fcc5510ed43c8b9c5a068c1768061678 "pdflatex"
+ "appendix.aux" 1719008557 4273 fcc5510ed43c8b9c5a068c1768061678 "pdflatex"
"appendix.tex" 1714902046 153 96de2ec39c79368bf559da3902b44c49 ""
"c:/Users/danie/Desktop/Masterarbbeit NLP/masterthesis.tex" 1718918656 3388 1a3d3b050246f34b1da3790d15078c72 ""
- "conclusion.aux" 1719004774 4615 4ded98a4a3c3a67bf9385588ceaa4c7a "pdflatex"
+ "conclusion.aux" 1719008557 4615 4ded98a4a3c3a67bf9385588ceaa4c7a "pdflatex"
"conclusion.tex" 1717528359 335 9748e8b1241fc403232a54d74006565d ""
- "dataset.aux" 1719004772 4620 41239dfac69f37589b36fea166d314e6 "pdflatex"
+ "dataset.aux" 1719008556 4620 41239dfac69f37589b36fea166d314e6 "pdflatex"
"dataset.tex" 1717528568 3652 939e7e589664966b254853117003c9a5 ""
- "experiments.aux" 1719004774 7277 bc90aae8838c76d1ee3c945ca4e4f5cf "pdflatex"
- "experiments.tex" 1719004765 32367 e5db99a534505a814a1f79b25d06b87e ""
- "introduction.aux" 1719004772 4279 8c6f0e498f6d68235a071aa1caa56107 "pdflatex"
+ "experiments.aux" 1719008557 7277 bc90aae8838c76d1ee3c945ca4e4f5cf "pdflatex"
+ "experiments.tex" 1719008553 32571 810c6a63ec03a3777784360400e90311 ""
+ "introduction.aux" 1719008555 4279 8c6f0e498f6d68235a071aa1caa56107 "pdflatex"
"introduction.tex" 1717514965 2519 caaf188bdca17cf9dc5a9a75a0711e3c ""
- "masterthesis.aux" 1719004774 7676 e718dbca29efdf0ff54b7c7021ec0c99 "pdflatex"
+ "masterthesis.aux" 1719008557 7676 870334d01829bdb87bd03fea6c061925 "pdflatex"
"masterthesis.bbl" 1718729075 15919 597efc0038b12521d31156c1f1e6bc00 "biber masterthesis"
- "masterthesis.out" 1719004771 0 d41d8cd98f00b204e9800998ecf8427e "pdflatex"
- "masterthesis.run.xml" 1719004774 2400 1590a67ffdbdc89a727a06496d3c1bae "pdflatex"
+ "masterthesis.out" 1719008555 0 d41d8cd98f00b204e9800998ecf8427e "pdflatex"
+ "masterthesis.run.xml" 1719008558 2400 1590a67ffdbdc89a727a06496d3c1bae "pdflatex"
"masterthesis.tex" 1718918656 3388 1a3d3b050246f34b1da3790d15078c72 ""
- "masterthesis.toc" 1719004774 2083 e252035d83121e27d02c87cd8ca1d146 "pdflatex"
- "methodology.aux" 1719004772 5564 ad289baffc6b9187af1ad6ccccdc7dd8 "pdflatex"
+ "masterthesis.toc" 1719008558 2083 e252035d83121e27d02c87cd8ca1d146 "pdflatex"
+ "methodology.aux" 1719008555 5564 ad289baffc6b9187af1ad6ccccdc7dd8 "pdflatex"
"methodology.tex" 1717708498 10014 b8c408b31a520ffbdfc632b9c71fef47 ""
- "related_work.aux" 1719004772 4280 6b78037eeba9c3cbd4c715c7100a50f0 "pdflatex"
+ "related_work.aux" 1719008556 4280 6b78037eeba9c3cbd4c715c7100a50f0 "pdflatex"
"related_work.tex" 1714902046 2004 9153b206d7152d9e4845d0e453956980 ""
"ressources/data/boxplot.csv" 1718372255 787 519a7447dc5f7bd6600c5df12532bd26 ""
"ressources/data/boxplot2.csv" 1718388614 793 a962dfffdce3eb67873136aaadd2d495 ""
diff --git a/masterthesis.log b/masterthesis.log
index d107cbadb1b640e9823f8f1206067eb2b970f2af..825a8e00ef89d769cd4edb2beffe0d4b544fcd2b 100644
--- a/masterthesis.log
+++ b/masterthesis.log
@@ -1,4 +1,4 @@
-This is pdfTeX, Version 3.141592653-2.6-1.40.26 (MiKTeX 24.4) (preloaded format=pdflatex 2024.5.6) 21 JUN 2024 23:19
+This is pdfTeX, Version 3.141592653-2.6-1.40.26 (MiKTeX 24.4) (preloaded format=pdflatex 2024.5.6) 22 JUN 2024 00:22
entering extended mode
restricted \write18 enabled.
file:line:error style messages enabled.
@@ -1466,21 +1466,21 @@ LaTeX Font Info: Font shape `U/msb/m/n' will be
Underfull \vbox (badness 10000) has occurred while \output is active []
[17]
-Underfull \hbox (badness 10000) in paragraph at lines 711--712
+Underfull \hbox (badness 10000) in paragraph at lines 710--711
[]
-Underfull \hbox (badness 10000) in paragraph at lines 713--714
+Underfull \hbox (badness 10000) in paragraph at lines 712--713
[]
-
-Underfull \hbox (badness 10000) in paragraph at lines 715--716
+[18]
+Underfull \hbox (badness 10000) in paragraph at lines 714--715
[]
-[18] [19]) [20]
+[19]) [20]
\openout2 = `conclusion.aux'.
(conclusion.tex
@@ -1538,13 +1538,13 @@ Package logreq Info: Writing requests to 'masterthesis.run.xml'.
Here is how much of TeX's memory you used:
50351 strings out of 474424
1195545 string characters out of 5741740
- 3355851 words of memory out of 5000000
+ 3355849 words of memory out of 5000000
71814 multiletter control sequences out of 15000+600000
598827 words of font info for 124 fonts, out of 8000000 for 9000
1145 hyphenation exceptions out of 8191
108i,20n,107p,10950b,2863s stack positions out of 10000i,1000n,20000p,200000b,200000s
<C:/Users/danie/AppData/Local/Programs/MiKTeX/fonts/type1/public/amsfonts/cm/cmex10.pfb><C:/Users/danie/AppData/Local/Programs/MiKTeX/fonts/type1/public/amsfonts/cm/cmmi10.pfb><C:/Users/danie/AppData/Local/Programs/MiKTeX/fonts/type1/public/amsfonts/cm/cmr10.pfb><C:/Users/danie/AppData/Local/Programs/MiKTeX/fonts/type1/public/amsfonts/cm/cmsy10.pfb><C:/Users/danie/AppData/Local/Programs/MiKTeX/fonts/type1/public/mathpazo/fplmr.pfb><C:/Users/danie/AppData/Local/Programs/MiKTeX/fonts/type1/public/mathpazo/fplmri.pfb><C:/Users/danie/AppData/Local/Programs/MiKTeX/fonts/type1/urw/courier/ucrr8a.pfb><C:/Users/danie/AppData/Local/Programs/MiKTeX/fonts/type1/urw/palatino/uplb8a.pfb><C:/Users/danie/AppData/Local/Programs/MiKTeX/fonts/type1/urw/palatino/uplr8a.pfb><C:/Users/danie/AppData/Local/Programs/MiKTeX/fonts/type1/urw/palatino/uplr8a.pfb><C:/Users/danie/AppData/Local/Programs/MiKTeX/fonts/type1/urw/palatino/uplri8a.pfb>
-Output written on masterthesis.pdf (31 pages, 660179 bytes).
+Output written on masterthesis.pdf (31 pages, 660457 bytes).
PDF statistics:
370 PDF objects out of 1000 (max. 8388607)
69 named destinations out of 1000 (max. 500000)
diff --git a/masterthesis.pdf b/masterthesis.pdf
index 08df48213bcfb76667bf01a046c037c8e67127c7..6062a239ee239486ea2488e1893a8ec11f4725ce 100644
Binary files a/masterthesis.pdf and b/masterthesis.pdf differ
diff --git a/masterthesis.synctex.gz b/masterthesis.synctex.gz
index 154d2762792bb873bd97e111fa5063a07216bd8e..93cb481ee99d7764da94a873a174c8f545f7cce3 100644
Binary files a/masterthesis.synctex.gz and b/masterthesis.synctex.gz differ