diff --git a/Morphilo_doc/_build/doctrees/environment.pickle b/Morphilo_doc/_build/doctrees/environment.pickle
index 07a3c8769109142ba03998aa4b347f13bf5c40ed..137a3bd7dab33aa052519cb0fb57672d7ddf3f2d 100644
Binary files a/Morphilo_doc/_build/doctrees/environment.pickle and b/Morphilo_doc/_build/doctrees/environment.pickle differ
diff --git a/Morphilo_doc/_build/doctrees/index.doctree b/Morphilo_doc/_build/doctrees/index.doctree
index b1ce5341c9ec85930100b86af5caa390ac58d26e..9f7397d8cdd78d89fa5ea3ecca2d0c26d9c57119 100644
Binary files a/Morphilo_doc/_build/doctrees/index.doctree and b/Morphilo_doc/_build/doctrees/index.doctree differ
diff --git a/Morphilo_doc/_build/doctrees/source/architecture.doctree b/Morphilo_doc/_build/doctrees/source/architecture.doctree
index 7b51612d552be51cbc60fd6394feb8b8a859c997..6cbb1d1e1885f84f7a17c9f5666374455c2e581d 100644
Binary files a/Morphilo_doc/_build/doctrees/source/architecture.doctree and b/Morphilo_doc/_build/doctrees/source/architecture.doctree differ
diff --git a/Morphilo_doc/_build/doctrees/source/controller.doctree b/Morphilo_doc/_build/doctrees/source/controller.doctree
index af065b89378efaf7a1237ecbada92a78dddf1af2..a0ab600ee7343ef5e8277e4d8a63870c78a49aba 100644
Binary files a/Morphilo_doc/_build/doctrees/source/controller.doctree and b/Morphilo_doc/_build/doctrees/source/controller.doctree differ
diff --git a/Morphilo_doc/_build/doctrees/source/datamodel.doctree b/Morphilo_doc/_build/doctrees/source/datamodel.doctree
index d9306dc94fb646762de7f705c1140572b16d39c5..7a02cd8edf6141f9db9b26cc1ee32f1c45272901 100644
Binary files a/Morphilo_doc/_build/doctrees/source/datamodel.doctree and b/Morphilo_doc/_build/doctrees/source/datamodel.doctree differ
diff --git a/Morphilo_doc/_build/doctrees/source/framework.doctree b/Morphilo_doc/_build/doctrees/source/framework.doctree
index 398e1c277effc4fa832fe35372ae81e56970296b..03befb26708ea73d4fd06ada1e79affd968f7def 100644
Binary files a/Morphilo_doc/_build/doctrees/source/framework.doctree and b/Morphilo_doc/_build/doctrees/source/framework.doctree differ
diff --git a/Morphilo_doc/_build/doctrees/source/view.doctree b/Morphilo_doc/_build/doctrees/source/view.doctree
index a2fb1ec70077bbaf14a994eace6594fb7c7a11cc..4848c152024c56f96670ee5e87735f125dae3045 100644
Binary files a/Morphilo_doc/_build/doctrees/source/view.doctree and b/Morphilo_doc/_build/doctrees/source/view.doctree differ
diff --git a/Morphilo_doc/_build/html/_images/120px-Green_eyes_kitten.jpg b/Morphilo_doc/_build/html/_images/120px-Green_eyes_kitten.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..b3c0a99b1c59cc198c270b0ebd4c1fb318ecc097
Binary files /dev/null and b/Morphilo_doc/_build/html/_images/120px-Green_eyes_kitten.jpg differ
diff --git a/Morphilo_doc/_build/html/_images/FotoHP2012.jpg b/Morphilo_doc/_build/html/_images/FotoHP2012.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..c8b3d4956087d8b75fafb076ee5c4ad251a6db9b
Binary files /dev/null and b/Morphilo_doc/_build/html/_images/FotoHP2012.jpg differ
diff --git a/Morphilo_doc/_build/html/_images/architecture.png b/Morphilo_doc/_build/html/_images/architecture.png
new file mode 100644
index 0000000000000000000000000000000000000000..0419932563c7b07aa600982cf0117881b200c50d
Binary files /dev/null and b/Morphilo_doc/_build/html/_images/architecture.png differ
diff --git a/Morphilo_doc/_build/html/_images/morphilo_uml.png b/Morphilo_doc/_build/html/_images/morphilo_uml.png
new file mode 100644
index 0000000000000000000000000000000000000000..cac16330b005f6d7e4e8d3e35d1e00c5839fe528
Binary files /dev/null and b/Morphilo_doc/_build/html/_images/morphilo_uml.png differ
diff --git a/Morphilo_doc/_build/html/_images/mycore_architecture-2.png b/Morphilo_doc/_build/html/_images/mycore_architecture-2.png
new file mode 100644
index 0000000000000000000000000000000000000000..5e8af028e9c209ceaedb5fd8bde7d97d3ee2fc1c
Binary files /dev/null and b/Morphilo_doc/_build/html/_images/mycore_architecture-2.png differ
diff --git a/Morphilo_doc/_build/html/_sources/index.rst.txt b/Morphilo_doc/_build/html/_sources/index.rst.txt
index c7043067b4c65dfe25c731ac0e2deb7221e09da4..466fbd75ad335baa0d368537c5a5e61d07652ab8 100644
--- a/Morphilo_doc/_build/html/_sources/index.rst.txt
+++ b/Morphilo_doc/_build/html/_sources/index.rst.txt
@@ -3,18 +3,18 @@
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
-Documentation Morphilo Project
-====================================
+Morphilo Project Documentation
+==============================
.. toctree::
:maxdepth: 3
:caption: Contents:
+ source/architecture.rst
source/datamodel.rst
- source/controller.rst
source/view.rst
- source/architecture.rst
- source/framework.rst
+ source/controller.rst
+
Indices and tables
==================
diff --git a/Morphilo_doc/_build/html/_sources/source/architecture.rst.txt b/Morphilo_doc/_build/html/_sources/source/architecture.rst.txt
index 5b114bda79b5dc2cca9d10df9d300507bf510317..1d26cbe25a19990c74a44c89ff2f374189ef7b22 100644
--- a/Morphilo_doc/_build/html/_sources/source/architecture.rst.txt
+++ b/Morphilo_doc/_build/html/_sources/source/architecture.rst.txt
@@ -1,56 +1,95 @@
Software Design
===============
+MVC Model
+---------
-.. image:: architecture.*
+A standard architecture for software has become a form of an
+observer pattern called *Model-View-Controller (MVC)*-Model [#f3]_.
+This is escpecially true for web-based applications that use
+some form of a client-server architecture since these systems naturally divide
+the browser view from the rest of the program logic and, if dynamically set up,
+also from the data model usually running in an extra server as well.
+As already implied, the MVC-pattern modularizes the program into three components: model, view, and
+controller coupled *low* by interfaces. The view is concerned with
+everything the actual user sees on the screen or uses to interact with the
+machine. The controller is to recognize and process the events initiated by the
+user and to update the view. Processing involves to communicate with the model.
+This may involve to save or provide data from the data base.
+From all that follows, MVC-models are especially supportive for reusing
+existing software and promotes parallel development of its three components.
+So the data model of an existing program can easily be changed without touching
+the essentials of the program logic. The same is true for the code that handles
+the view. Most of the time view and data model are the two components that need
+to be changed so that the software appearance and presentation is adjusted to
+the new user group as well as the different data is adjusted to the needs of the different
+requirements of the new application. Nevertheless, if bugs or general changes in
+the controller component have to be done, it usually does not affect
+substantially the view and data model.
-The architecture of a possible **take-and-share**-approach for language
-resources is visualized in figure \ref{fig:architect}. Because the very gist
+Another positive consequence of MVC-models is that several views (or even
+models) could be used simultaneously. It means that the same data could be
+presented differently on the user interface.
+
+
+Morphilo Architecture
+---------------------
+
+
+.. figure:: images/architecture.png
+
+ Figure 1: Basic Architecture of a Take-&-Share-Approach
+
+
+The architecture of a possible *take-and-share* approach for language
+resources is visualized in figure 1. Because the very gist
of the approach becomes clearer if describing a concrete example, the case of
-annotating lexical derivatives of Middle English and a respective database is
-given as an illustration.
+annotating lexical derivatives of Middle English with the help of the Morphilo Tool
+[#f1]_ using a `MyCoRe repository <http://www.mycore.de>`_ is given as an illustration.
However, any other tool that helps with manual annotations and manages metadata of a corpus could be
-substituted here instead.
+substituted here instead. [#f2]_
After inputting an untagged corpus or plain text, it is determined whether the
input material was annotated previously by a different user. This information is
usually provided by the metadata administered by the annotation tool; in the case at
-hand it is called \emph{Morphilizer} in figure \ref{fig:architect}. An
-alternative is a simple table look-up for all occurring words in the datasets Corpus 1 through Corpus n. If contained
-completely, the \emph{yes}-branch is followed up further -- otherwise \emph{no}
+hand, the *Morphilo* component. An alternative is a
+simple table look-up for all occurring words in the datasets Corpus 1 through Corpus n. If contained
+completely, the *yes*-branch is followed up further -- otherwise *no*
succeeds. The difference between the two branches is subtle, yet crucial. On
-both branches, the annotation tool (here \emph{Morphilizer}) is called, which, first,
-sorts out all words that are not contained in the master database (here \emph{Morphilo-DB})
+both branches, the annotation tool (here *Morphilo*) is called, which, first,
+sorts out all words that are not contained in the master database (here *MyCoRe* repository)
and, second, makes reasonable suggestions on an optimal annotation of
-the items. In both cases the
-annotations are linked to the respective items (e.g. words) in the
-text, but they are also persistently saved in an extra dataset, i.e. Corpus 1
+the items. The suggestions made to the user are based on simple string mapping of a saved list of prefixes and suffixes
+whereas the remainder of the mapping is defined as the word root. The annotations are linked to the respective items (e.g. words) in the
+text, but they are also persistently saved in an extra dataset, i.e. in figure 1 in one of the delineated Corpus 1
through n, together with all available metadata.
-The difference between both information streams is that
-in the \emph{yes}-branch a comparison between the newly created dataset and
-all of the previous datasets of this text is carried out. Within this
-unit, all deviations and congruencies are marked and counted. The underlying
+The difference between the two branches in figure 1 is that
+in the *yes*-branch a comparison between the newly created dataset and
+all of the previous datasets of this text is carried out while this is not
+possible if a text was not annotated before. Within this
+unit, all deviations and congruencies of the annotated items are marked and counted. The underlying
assumption is that with a growing number of comparable texts the
correct annotations approach a theoretic true value of a correct annotation
while errors level out provided that the sample size is large enough. How the
distribution of errors and correct annotations exactly looks like and if a
normal distribution can be assumed is still object of the ongoing research, but
-independent of the concrete results, the component (called \emph{compare
-manual annotations} in figure \ref{fig:architect}) allows for specifying the
+independent of the concrete results, the component (called *compare
+manual annotations* in figure 1) allows for specifying the
exact form of the sample population.
In fact, it is necessary at that point to define the form of the distribution,
-sample size, and the rejection region. The standard setting are a normal
-distribution, a rejection region of $\alpha = 0.05$ and sample size of $30$ so
-that a simple Gau\ss-Test can be calculated.
+sample size, and the rejection region. To be put it simple here, a uniform distribution in form of a threshold value
+of e.g. 20 could be defined that specifies that a word has to be annotated equally by
+20 different users before it enters the master database.
-Continuing the information flow further, these statistical calculations are
+Continuing the information flow in figure 1 further, the threshold values or, if so defined,
+the results of the statistical calculation of other distributions respectively are
delivered to the quality-control-component. Based on the statistics, the
respective items together with the metadata, frequencies, and, of course,
annotations are written to the master database. All information in the master
database is directly used for automated annotations. Thus it is directly matched
-to the input texts or corpora respectively through the \emph{Morphilizer}-tool.
+to the input texts or corpora respectively through the *Morphilo*-tool.
The annotation tool decides on the entries looked up in the master which items
are to be manually annotated.
@@ -63,4 +102,35 @@ possible to interrupt the annotation work and save progress on the server. And
the user will have access to the annotations made in the respective dataset,
correct them or save them and resume later. It is important to note that the user will receive
the tagged document only after all items are fully annotated. No partially
-tagged text can be output.
\ No newline at end of file
+tagged text can be output.
+
+Repository Framework
+--------------------
+
+.. figure:: images/mycore_architecture-2.png
+
+ Figure 2: `MyCoRe <http://www.mycore.de>`_-Architecture and Components
+
+To specify the repository framework, the morphilo application logic will have to be implemented,
+a data model specified, and the input, search and output mask programmed.
+
+There are three directories which are
+important for adjusting the MyCoRe framework to the needs of one's own application.
+
+These three directories
+correspond essentially to the three components in the MVC model as explicated above. Roughly, they are also envisualized in figure 2 in the upper
+right hand corner. More precisely, the view (*Layout* in figure 2) and the model layer
+(*Datenmodell* in figure 2) can be done
+completely via the *interface*, which is a directory with a predefined
+structure and some standard files. For the configuration of the logic an extra directory is offered (*/src/main/java/custom/mycore/addons/*). Here all, java classes
+extending the controller layer should be added.
+Practically, all three MVC layers are placed in the
+*src/main/*-directory of the application. In one of the subdirectories,
+*datamodel/def,* the datamodel specifications are defined as xml files. It parallels the model
+layer in the MVC pattern. How the data model was defined will be explained in the section Data Model.
+
+.. rubric:: Notes
+
+.. [#f1] Peukert, H. (2012): From Semi-Automatic to Automatic Affix Extraction in Middle English Corpora: Building a Sustainable Database for Analyzing Derivational Morphology over Time, Empirical Methods in Natural Language Processing, Wien, Scientific series of the ÖGAI, 413-23.
+.. [#f2] The source code of a possible implementation is available on https://github.com/amadeusgwin/morphilo. The software runs in test mode on https://www.morphilo.uni-hamburg.de/content/index.xml.
+.. [#f3] Butz, Andreas; Antonio Krüger (2017): Mensch-Maschine-Interaktion, De Gruyter, 93ff.
\ No newline at end of file
diff --git a/Morphilo_doc/_build/html/_sources/source/controller.rst.txt b/Morphilo_doc/_build/html/_sources/source/controller.rst.txt
index 6f6b896272e6cade94e54ff34fd76af6bfc28326..e63e0e81c05dd42f0aed235e5467e316d70ca96e 100644
--- a/Morphilo_doc/_build/html/_sources/source/controller.rst.txt
+++ b/Morphilo_doc/_build/html/_sources/source/controller.rst.txt
@@ -4,11 +4,18 @@ Controller Adjustments
General Principle of Operation
------------------------------
-Figure \ref{fig:classDiag} illustrates the dependencies of the five java classes that were integrated to add the morphilo
-functionality defined in the default package \emph{custom.mycore.addons.morphilo}. The general principle of operation
+.. _classDiag:
+
+.. figure:: images/morphilo_uml.png
+
+ Figure 3: Morphilo UML Diagramm
+
+
+Figure :ref:`classDiag` illustrates the dependencies of the five java classes that were integrated to add the morphilo
+functionality defined in the default package *custom.mycore.addons.morphilo*. The general principle of operation
is the following. The handling of data search, upload, saving, and user
authentification is fully left to the MyCoRe functionality that is completely
-implemented. The class \emph{ProcessCorpusServlet.java} receives a request from the webinterface to process an uploaded file,
+implemented. The class *ProcessCorpusServlet.java* receives a request from the webinterface to process an uploaded file,
i.e. a simple text corpus, and it checks if any of the words are available in the master database. All words that are not
listed in the master database are written to an extra file. These are the words that have to be manually annotated. At the end, the
servlet sends a response back to the user interface. In case of all words are contained in the master, an xml file is generated from the
@@ -16,38 +23,29 @@ master database that includes all annotated words of the original corpus. Usuall
So if some words are not in the master, the user will get the response to initiate the manual annotation process.
The manual annotation process is processed by the class
-\emph{{Tag\-Corpus\-Serv\-let\-.ja\-va}}, which will build a JDOM object for the first word in the extra file.
-This is done by creating an object of the \emph{JDOMorphilo.java} class. This class, in turn, will use the methods of
-\emph{AffixStripper.java} that make simple, but reasonable, suggestions on the word structure. This JDOM object is then
+*TagCorpusServlet.java*, which will build a JDOM object for the first word in the extra file.
+This is done by creating an object of the *JDOMorphilo.java* class. This class, in turn, will use the methods of
+*AffixStripper.java* that make simple, but reasonable, suggestions on the word structure. This JDOM object is then
given as a response back to the user. It is presented as a form, in which the user can make changes. This is necessary
-because the word structure algorithm of \emph{AffixStripper.java} errs in some cases. Once the user agrees on the
+because the word structure algorithm of *AffixStripper.java* errs in some cases. Once the user agrees on the
suggestions or on his or her corrections, the JDOM object is saved as an xml that is only searchable, visible, and
changeable by the authenicated user (and the administrator), another file containing all processed words is created or
-updated respectively and the \emph{TagCorpusServlet.java} servlet will restart until the last word in the extra list is
+updated respectively and the *TagCorpusServlet.java* servlet will restart until the last word in the extra list is
processed. This enables the user to stop and resume her or his annotation work at a later point in time. The
-\emph{TagCorpusServlet} will call methods from \emph{ProcessCorpusServlet.java} to adjust the content of the extra
+*TagCorpusServlet* will call methods from *ProcessCorpusServlet.java* to adjust the content of the extra
files harboring the untagged words. If this file is empty, and only then, it is replaced by the file comprising all words
from the original text file, both the ones from the master database and the ones that are annotated by the user,
in an annotated xml representation.
-Each time \emph{ProcessCorpusServlet.java} is instantiated, it also instantiates \emph{QualityControl.java}. This class checks if a
+Each time *ProcessCorpusServlet.java* is instantiated, it also instantiates *QualityControl.java*. This class checks if a
new word can be transferred to the master database. The algorithm can be freely adopted to higher or lower quality standards.
In its present configuration, a method tests at a limit of 20 different
registered users agreeing on the annotation of the same word. More specifically,
-if 20 JDOM objects are identical except in the attribute field \emph{occurrences} in the metadata node, the JDOM object becomes
-part of the master. The latter is easily done by changing the attribute \emph{creator} from the user name
-to \emph{``administrator''} in the service node. This makes the dataset part of the master database. Moreover, the \emph{occurrences}
+if 20 JDOM objects are identical except in the attribute field *occurrences* in the metadata node, the JDOM object becomes
+part of the master. The latter is easily done by changing the attribute *creator* from the user name
+to *administrator* in the service node. This makes the dataset part of the master database. Moreover, the *occurrences*
attribute is updated by adding up all occurrences of the word that stem from
different text corpora of the same time range.
-\begin{landscape}
- \begin{figure}
- \centering
- \includegraphics[scale=0.55]{morphilo_uml.png}
- \caption{Class Diagram Morphilo}
- \label{fig:classDiag}
- \end{figure}
-\end{landscape}
-
Conceptualization
@@ -60,533 +58,572 @@ files as saving, creating, deleting, and updating files.
Moreover, a rudimentary user management comprising different roles and
rights is offered. The basic technology behind the controller's logic is the
servlet. As such all new code has to be registered as a servlet in the
-web-fragment.xml (here the Apache Tomcat container) as listing \ref{lst:webfragment} shows.
-
-\begin{lstlisting}[language=XML,caption={Servlet Registering in the
-web-fragment.xml (excerpt)},label=lst:webfragment,escapechar=|]
-<servlet>
- <servlet-name>ProcessCorpusServlet</servlet-name>
- <servlet-class>custom.mycore.addons.morphilo.ProcessCorpusServlet</servlet-class>
-</servlet>
-<servlet-mapping>
- <servlet-name>ProcessCorpusServlet</servlet-name>
- <url-pattern>/servlets/object/process</url-pattern>|\label{ln:process}|
-</servlet-mapping>
-<servlet>
- <servlet-name>TagCorpusServlet</servlet-name>
- <servlet-class>custom.mycore.addons.morphilo.TagCorpusServlet</servlet-class>
-</servlet>
-<servlet-mapping>
- <servlet-name>TagCorpusServlet</servlet-name>
- <url-pattern>/servlets/object/tag</url-pattern>|\label{ln:tag}|
-</servlet-mapping>
-\end{lstlisting}
-
-Now, the logic has to be extended by the specifications analyzed in chapter
-\ref{chap:concept} on conceptualization. More specifically, some
+web-fragment.xml (here the Apache Tomcat container).
+
+.. _webxml:
+
+.. code-block:: xml
+ :caption: Servlet Registering in the web-fragment.xml
+ :linenos:
+ :emphasize-lines: 7, 15
+
+ <servlet>
+ <servlet-name>ProcessCorpusServlet</servlet-name>
+ <servlet-class>custom.mycore.addons.morphilo.ProcessCorpusServlet</servlet-class>
+ </servlet>
+ <servlet-mapping>
+ <servlet-name>ProcessCorpusServlet</servlet-name>
+ <url-pattern>/servlets/object/process</url-pattern>
+ </servlet-mapping>
+ <servlet>
+ <servlet-name>TagCorpusServlet</servlet-name>
+ <servlet-class>custom.mycore.addons.morphilo.TagCorpusServlet</servlet-class>
+ </servlet>
+ <servlet-mapping>
+ <servlet-name>TagCorpusServlet</servlet-name>
+ <url-pattern>/servlets/object/tag</url-pattern>
+ </servlet-mapping>
+ \end{lstlisting}
+
+Now, the logic has to be extended by the specifications. Some
classes have to be added that take care of analyzing words
-(\emph{AffixStripper.java, InflectionEnum.java, SuffixEnum.java,
-PrefixEnum.java}), extracting the relevant words from the text and checking the
-uniqueness of the text (\emph{ProcessCorpusServlet.java}), make reasonable
-suggestions on the annotation (\emph{TagCorpusServlet.java}), build the object
-of each annotated word (\emph{JDOMorphilo.java}), and check on the quality by applying
-statistical models (\emph{QualityControl.java}).
+(*AffixStripper.java, InflectionEnum.java, SuffixEnum.java,
+PrefixEnum.java*), extracting the relevant words from the text and checking the
+uniqueness of the text (*ProcessCorpusServlet.java*), make reasonable
+suggestions on the annotation (*TagCorpusServlet.java*), build the object
+of each annotated word (*JDOMorphilo.java*), and check on the quality by applying
+statistical models (*QualityControl.java*).
+
+.. _controller-section:
Implementation
--------------
Having taken a bird's eye perspective in the previous chapter, it is now time to take a look at the specific implementation at the level
-of methods. Starting with the main servlet, \emph{ProcessCorpusServlet.java}, the class defines four getter method:
-\renewcommand{\labelenumi}{(\theenumi)}
-\begin{enumerate}
- \item\label{itm:geturl} public String getURLParameter(MCRServletJob, String)
- \item\label{itm:getcorp} public String getCorpusMetadata(MCRServletJob, String)
- \item\label{itm:getcont} public ArrayList<String> getContentFromFile(MCRServletJob, String)
- \item\label{itm:getderiv} public Path getDerivateFilePath(MCRServletJob, String)
- \item\label{itm:now} public int getNumberOfWords(MCRServletJob job, String)
-\end{enumerate}
+of methods. Starting with the main servlet, *ProcessCorpusServlet.java*, the class defines four getter method:
+
+#. *public String getURLParameter(MCRServletJob, String)*
+#. *public String getCorpusMetadata(MCRServletJob, String)*
+#. *public ArrayList<String> getContentFromFile(MCRServletJob, String)*
+#. *public Path getDerivateFilePath(MCRServletJob, String)*
+#. *public int getNumberOfWords(MCRServletJob job, String)*
+
Since each servlet in MyCoRe extends the class MCRServlet, it has access to MCRServletJob, from which the http requests and responses
-can be used. This is the first argument in the above methods. The second argument of method (\ref{itm:geturl}) specifies the name of an url parameter, i.e.
-the object id or the id of the derivate. The method returns the value of the given parameter. Typically MyCoRe uses the url to exchange
+can be used. This is the first argument in the above methods. The second argument of method (in 1.) specifies the name of an url parameter, i.e.
+the object id or the id of the derivate. The method returns the value of the given parameter. Typically *MyCoRe* uses the url to exchange
these ids. The second method provides us with the value of a data field in the xml document. So the string defines the name of an attribute.
-\emph{getContentFromFile(MCRServletJob, String)} returns the words as a list from a file when given the filename as a string.
-The getter listed in \ref{itm:getderiv}), returns the Path from the MyCoRe repository when the name of
-the file is specified. And finally, method (\ref{itm:now}) returns the number of words by simply returning
-\emph{getContentFromFile(job, fileName).size()}.
+*getContentFromFile(MCRServletJob, String)* returns the words as a list from a file when given the filename as a string.
+The getter listed in 4., returns the Path from the MyCoRe repository when the name of
+the file is specified. And finally, method (in 5.) returns the number of words by simply returning
+*getContentFromFile(job, fileName).size()*.
There are two methods in every MyCoRe-Servlet that have to be overwritten,
-\emph{protected void render(MCRServletJob, Exception)}, which redirects the requests as \emph{POST} or \emph{GET} responds, and
-\emph{protected void think(MCRServletJob)}, in which the logic is implemented. Since the latter is important to understand the
-core idea of the Morphilo algorithm, it is displayed in full length in source code \ref{src:think}.
-
-\begin{lstlisting}[language=java,caption={The overwritten think method},label=src:think,escapechar=|]
-protected void think(MCRServletJob job) throws Exception
-{
- this.job = job;
- String dateFromCorp = getCorpusMetadata(job, "def.datefrom");
- String dateUntilCorp = getCorpusMetadata(job, "def.dateuntil");
- String corpID = getURLParameter(job, "objID");
- String derivID = getURLParameter(job, "id");
-
- //if NoW is 0, fill with anzWords
- MCRObject helpObj = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(corpID));|\label{ln:bugfixstart}|
- Document jdomDocHelp = helpObj.createXML();
- XPathFactory xpfacty = XPathFactory.instance();
- XPathExpression<Element> xpExp = xpfacty.compile("//NoW", Filters.element());
- Element elem = xpExp.evaluateFirst(jdomDocHelp);
- //fixes transferred morphilo data from previous stand alone project
- int corpussize = getNumberOfWords(job, "");
- if (Integer.parseInt(elem.getText()) != corpussize)
- {
- elem.setText(Integer.toString(corpussize));
- helpObj = new MCRObject(jdomDocHelp);
- MCRMetadataManager.update(helpObj);
- }|\label{ln:bugfixend}|
-
- //Check if the uploaded corpus was processed before
- SolrClient slr = MCRSolrClientFactory.getSolrClient();|\label{ln:solrstart}|
- SolrQuery qry = new SolrQuery();
- qry.setFields("korpusname", "datefrom", "dateuntil", "NoW", "id");
- qry.setQuery("datefrom:" + dateFromCorp + " AND dateuntil:" + dateUntilCorp + " AND NoW:" + corpussize);
- SolrDocumentList rslt = slr.query(qry).getResults();|\label{ln:solrresult}|
-
- Boolean incrOcc = true;
- // if resultset contains only one, then it must be the newly created corpus
- if (slr.query(qry).getResults().getNumFound() > 1)
- {
- incrOcc = false;
- }|\label{ln:solrend}|
-
- //match all words in corpus with morphilo (creator=administrator) and save all words that are not in morphilo DB in leftovers
- ArrayList<String> leftovers = new ArrayList<String>();
- ArrayList<String> processed = new ArrayList<String>();
-
- leftovers = getUnknownWords(getContentFromFile(job, ""), dateFromCorp, dateUntilCorp, "", incrOcc, incrOcc, false);|\label{ln:callkeymeth}|
-
- //write all words of leftover in file as derivative to respective corpmeta dataset
- MCRPath root = MCRPath.getPath(derivID, "/");|\label{ln:filesavestart}|
- Path fn = getDerivateFilePath(job, "").getFileName();
- Path p = root.resolve("untagged-" + fn);
- Files.write(p, leftovers);|\label{ln:filesaveend}|
-
- //create a file for all words that were processed
- Path procWds = root.resolve("processed-" + fn);
- Files.write(procWds, processed);
-}
-\end{lstlisting}
-Using the above mentioned getter methods, the \emph{think} method assigns values to the object ID, needed to get the xml document
-that contain the corpus metadata, the file ID, and the beginning and starting dates from the corpus to be analyzed. Lines \ref{ln:bugfixstart}
-through \ref{ln:bugfixend} show how to access a mycore object as an xml document, a procedure that will be used in different variants
+*protected void render(MCRServletJob, Exception)*, which redirects the requests as *POST* or *GET* responds, and
+*protected void think(MCRServletJob)*, in which the logic is implemented. Since the latter is important to understand the
+core idea of the Morphilo algorithm, it is displayed in full length in source code :ref:`think`.
+
+.. _think:
+
+.. code-block:: java
+ :caption: The overwritten think method
+ :linenos:
+ :emphasize-lines: 10, 22, 25, 36, 42, 45, 48
+
+ protected void think(MCRServletJob job) throws Exception
+ {
+ this.job = job;
+ String dateFromCorp = getCorpusMetadata(job, "def.datefrom");
+ String dateUntilCorp = getCorpusMetadata(job, "def.dateuntil");
+ String corpID = getURLParameter(job, "objID");
+ String derivID = getURLParameter(job, "id");
+
+ //if NoW is 0, fill with anzWords
+ MCRObject helpObj = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(corpID));
+ Document jdomDocHelp = helpObj.createXML();
+ XPathFactory xpfacty = XPathFactory.instance();
+ XPathExpression<Element> xpExp = xpfacty.compile("//NoW", Filters.element());
+ Element elem = xpExp.evaluateFirst(jdomDocHelp);
+ //fixes transferred morphilo data from previous stand alone project
+ int corpussize = getNumberOfWords(job, "");
+ if (Integer.parseInt(elem.getText()) != corpussize)
+ {
+ elem.setText(Integer.toString(corpussize));
+ helpObj = new MCRObject(jdomDocHelp);
+ MCRMetadataManager.update(helpObj);
+ }
+
+ //Check if the uploaded corpus was processed before
+ SolrClient slr = MCRSolrClientFactory.getSolrClient();
+ SolrQuery qry = new SolrQuery();
+ qry.setFields("korpusname", "datefrom", "dateuntil", "NoW", "id");
+ qry.setQuery("datefrom:" + dateFromCorp + " AND dateuntil:" + dateUntilCorp + " AND NoW:" + corpussize);
+ SolrDocumentList rslt = slr.query(qry).getResults();
+
+ Boolean incrOcc = true;
+ // if resultset contains only one, then it must be the newly created corpus
+ if (slr.query(qry).getResults().getNumFound() > 1)
+ {
+ incrOcc = false;
+ }
+
+ //match all words in corpus with morphilo (creator=administrator) and save all words that are not in morphilo DB in leftovers
+ ArrayList<String> leftovers = new ArrayList<String>();
+ ArrayList<String> processed = new ArrayList<String>();
+
+ leftovers = getUnknownWords(getContentFromFile(job, ""), dateFromCorp, dateUntilCorp, "", incrOcc, incrOcc, false);
+
+ //write all words of leftover in file as derivative to respective corpmeta dataset
+ MCRPath root = MCRPath.getPath(derivID, "/");
+ Path fn = getDerivateFilePath(job, "").getFileName();
+ Path p = root.resolve("untagged-" + fn);
+ Files.write(p, leftovers);
+
+ //create a file for all words that were processed
+ Path procWds = root.resolve("processed-" + fn);
+ Files.write(procWds, processed);
+ }
+
+Using the above mentioned getter methods, the *think*-method assigns values to the object ID, needed to get the xml document
+that contains the corpus metadata, the file ID, and the beginning and starting dates from the corpus to be analyzed. Lines 10
+through 22 show how to access a mycore object as an xml document, a procedure that will be used in different variants
throughout this implementation.
By means of the object ID, the respective corpus is identified and a JDOM document is constructed, which can then be accessed
by XPath. The XPath factory instances are collections of the xml nodes. In the present case, it is save to assume that only one element
-of \emph{NoW} is available (see corpus datamodel listing \ref{lst:corpusdatamodel} with $maxOccurs='1'$). So we do not have to loop through
-the collection, but use the first node named \emph{NoW}. The if-test checks if the number of words of the uploaded file is the
+of *NoW* is available (see corpus datamodel listing :ref:`corpusdatamodel` with *maxOccurs='1'*). So we do not have to loop through
+the collection, but use the first node named *NoW*. The if-test checks if the number of words of the uploaded file is the
same as the number written in the document. When the document is initially created by the MyCoRe logic it was configured to be zero.
-If unequal, the setText(String) method is used to write the number of words of the corpus to the document.
+If unequal, the *setText(String)* method is used to write the number of words of the corpus to the document.
-Lines \ref{ln:solrstart}--\ref{ln:solrend} reveal the second important ingredient, i.e. controlling the search engine. First, a solr
+Lines 25--36 reveal the second important ingredient, i.e. controlling the search engine. First, a solr
client and a query are initialized. Then, the output of the result set is defined by giving the fields of interest of the document.
-In the case at hand, it is the id, the name of the corpus, the number of words, and the beginnig and ending dates. With \emph{setQuery}
-it is possible to assign values to some or all of these fields. Finally, \emph{getResults()} carries out the search and writes
-all hits to a \emph{SolrDocumentList} (line \ref{ln:solrresult}). The test that follows is really only to set a Boolean
+In the case at hand, it is the id, the name of the corpus, the number of words, and the beginnig and ending dates. With *setQuery*
+it is possible to assign values to some or all of these fields. Finally, *getResults()* carries out the search and writes
+all hits to a *SolrDocumentList* (line 29). The test that follows is really only to set a Boolean
encoding if the number of occurrences of that word in the master should be updated. To avoid multiple counts,
incrementing the word frequency is only done if it is a new corpus.
-In line \ref{ln:callkeymeth} \emph{getUnknownWords(ArrayList, String, String, String, Boolean, Boolean, Boolean)} is called and
+In line 42 *getUnknownWords(ArrayList, String, String, String, Boolean, Boolean, Boolean)* is called and
returned as a list of words. This method is key and will be discussed in depth below. Finally, lines
-\ref{ln:filesavestart}--\ref{ln:filesaveend} show how to handle file objects in MyCoRe. Using the file ID, the root path and the name
-of the first file in that path are identified. Then, a second file starting with ``untagged'' is created and all words returned from
-the \emph{getUnknownWords} is written to that file. By the same token an empty file is created (in the last two lines of the \emph{think}-method),
+45--48 show how to handle file objects in MyCoRe. Using the file ID, the root path and the name
+of the first file in that path are identified. Then, a second file starting with *untagged* is created and all words returned from
+the *getUnknownWords* is written to that file. By the same token an empty file is created (in the last two lines of the *think*-method),
in which all words that are manually annotated will be saved.
-In a refactoring phase, the method \emph{getUnknownWords(ArrayList, String, String, String, Boolean, Boolean, Boolean)} could be subdivided into
+In a refactoring phase, the method *getUnknownWords(ArrayList, String, String, String, Boolean, Boolean, Boolean)* could be subdivided into
three methods: for each Boolean parameter one. In fact, this method handles more than one task. This is mainly due to multiple code avoidance.
-%this is just wrong because no resultset will substantially be more than 10-20
-%In addition, for large text files this method would run into efficiency problems if the master database also reaches the intended size of about
-%$100,000$ entries and beyond because
In essence, an outer loop runs through all words of the corpus and an inner loop runs through all hits in the solr result set. Because the result
-set is supposed to be small, approximately between $10-20$ items, efficiency
+set is supposed to be small, approximately between 10-20 items, efficiency
problems are unlikely to cause a problem, although there are some more loops running through collection of about the same sizes.
-%As the hits naturally grow larger with an increasing size of the data base, processing time will rise exponentially.
Since each word is identified on the basis of its projected word type, the word form, and the time range it falls into, it is these variables that
have to be checked for existence in the documents. If not in the xml documents,
-\emph{null} is returned and needs to be corrected. Moreover, user authentification must be considered. There are three different XPaths that are relevant.
-\begin{itemize}
- \item[-] \emph{//service/servflags/servflag[@type='createdby']} to test for the correct user
- \item[-] \emph{//morphiloContainer/morphilo} to create the annotated document
- \item[-] \emph{//morphiloContainer/morphilo/w} to set occurrences or add a link
-\end{itemize}
-
-As an illustration of the core functioning of this method, listing \ref{src:getUnknowWords} is given.
-\begin{lstlisting}[language=java,caption={Mode of Operation of getUnknownWords Method},label=src:getUnknowWords,escapechar=|]
-public ArrayList<String> getUnknownWords(
- ArrayList<String> corpus,
- String timeCorpusBegin,
- String timeCorpusEnd,
- String wdtpe,
- Boolean setOcc,
- Boolean setXlink,
- Boolean writeAllData) throws Exception
- {
- String currentUser = MCRSessionMgr.getCurrentSession().getUserInformation().getUserID();
- ArrayList lo = new ArrayList();
-
- for (int i = 0; i < corpus.size(); i++)
- {
- SolrClient solrClient = MCRSolrClientFactory.getSolrClient();
- SolrQuery query = new SolrQuery();
- query.setFields("w","occurrence","begin","end", "id", "wordtype");
- query.setQuery(corpus.get(i));
- query.setRows(50); //more than 50 items are extremely unlikely
- SolrDocumentList results = solrClient.query(query).getResults();
- Boolean available = false;
- for (int entryNum = 0; entryNum < results.size(); entryNum++)
- {
- ...
- // update in MCRMetaDataManager
- String mcrIDString = results.get(entryNum).getFieldValue("id").toString();
- //MCRObjekt auslesen und JDOM-Document erzeugen:
- MCRObject mcrObj = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(mcrIDString));
- Document jdomDoc = mcrObj.createXML();
- ...
- //check and correction for word type
- ...
- //checkand correction time: timeCorrect
- ...
- //check if user correct: isAuthorized
- ...
- XPathExpression<Element> xp = xpfac.compile("//morphiloContainer/morphilo/w", Filters.element());
- //Iterates w-elements and increments occurrence attribute if setOcc is true
- for (Element e : xp.evaluate(jdomDoc))
+*null* is returned and needs to be corrected. Moreover, user authentification must be considered. There are three different XPaths that are relevant.
+
+* *//service/servflags/servflag[@type='createdby']* to test for the correct user
+* *//morphiloContainer/morphilo* to create the annotated document
+* *//morphiloContainer/morphilo/w* to set occurrences or add a link
+
+As an illustration of the core functioning of this method, listing :ref:`getUnknowWords` is given.
+
+.. _getUnknowWords:
+
+.. code-block:: java
+ :caption: Mode of Operation of getUnknownWords Method
+ :linenos:
+ :emphasize-lines: 47, 60, 67, 69, 77
+
+ public ArrayList<String> getUnknownWords(
+ ArrayList<String> corpus,
+ String timeCorpusBegin,
+ String timeCorpusEnd,
+ String wdtpe,
+ Boolean setOcc,
+ Boolean setXlink,
+ Boolean writeAllData) throws Exception
{
- //wenn Rechte da sind und Worttyp nirgends gegeben oder gleich ist
- if (isAuthorized && timeCorrect
- && ((e.getAttributeValue("wordtype") == null && wdtpe.equals(""))
- || e.getAttributeValue("wordtype").equals(wordtype))) // nur zur Vereinheitlichung
+ String currentUser = MCRSessionMgr.getCurrentSession().getUserInformation().getUserID();
+ ArrayList lo = new ArrayList();
+
+ for (int i = 0; i < corpus.size(); i++)
+ {
+ SolrClient solrClient = MCRSolrClientFactory.getSolrClient();
+ SolrQuery query = new SolrQuery();
+ query.setFields("w","occurrence","begin","end", "id", "wordtype");
+ query.setQuery(corpus.get(i));
+ query.setRows(50); //more than 50 items are extremely unlikely
+ SolrDocumentList results = solrClient.query(query).getResults();
+ Boolean available = false;
+ for (int entryNum = 0; entryNum < results.size(); entryNum++)
{
- int oc = -1;
- available = true;|\label{ln:available}|
- try
- {
- //adjust occurrence Attribut
- if (setOcc)
+ ...
+ // update in MCRMetaDataManager
+ String mcrIDString = results.get(entryNum).getFieldValue("id").toString();
+ //MCRObjekt auslesen und JDOM-Document erzeugen:
+ MCRObject mcrObj = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(mcrIDString));
+ Document jdomDoc = mcrObj.createXML();
+ ...
+ //check and correction for word type
+ ...
+ //checkand correction time: timeCorrect
+ ...
+ //check if user correct: isAuthorized
+ ...
+ XPathExpression<Element> xp = xpfac.compile("//morphiloContainer/morphilo/w", Filters.element());
+ //Iterates w-elements and increments occurrence attribute if setOcc is true
+ for (Element e : xp.evaluate(jdomDoc))
+ {
+ //wenn Rechte da sind und Worttyp nirgends gegeben oder gleich ist
+ if (isAuthorized && timeCorrect
+ && ((e.getAttributeValue("wordtype") == null && wdtpe.equals(""))
+ || e.getAttributeValue("wordtype").equals(wordtype))) // nur zur Vereinheitlichung
{
- oc = Integer.parseInt(e.getAttributeValue("occurrence"));
- e.setAttribute("occurrence", Integer.toString(oc + 1));
- }
-
- //write morphilo-ObjectID in xml of corpmeta
- if (setXlink)
- {
- Namespace xlinkNamespace = Namespace.getNamespace("xlink", "http://www.w3.org/1999/xlink");|\label{ln:namespace}|
- MCRObject corpObj = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(getURLParameter(job, "objID")));
- Document corpDoc = corpObj.createXML();
- XPathExpression<Element> xpathEx = xpfac.compile("//corpuslink", Filters.element());
- Element elm = xpathEx.evaluateFirst(corpDoc);
- elm.setAttribute("href" , mcrIDString, xlinkNamespace);
- }
- mcrObj = new MCRObject(jdomDoc);|\label{ln:updatestart}|
- MCRMetadataManager.update(mcrObj);
- QualityControl qc = new QualityControl(mcrObj);|\label{ln:updateend}|
- }
- catch(NumberFormatException except)
- {
- // ignore
- }
- }
- }
- if (!available) // if not available in datasets under the given conditions |\label{ln:notavailable}|
- {
- lo.add(corpus.get(i));
- }
- }
- return lo;
- }
-\end{lstlisting}
-As can be seen from the functionality of listing \ref{src:getUnknowWords}, getting the unknown words of a corpus, is rather a side effect for the equally named method.
-More precisely, a Boolean (line \ref{ln:available}) is set when the document is manipulated otherwise because it is clear that the word must exist then.
-If the Boolean remains false (line \ref{ln:notavailable}), the word is put on the list of words that have to be annotated manually. As already explained above, the
+ int oc = -1;
+ available = true;
+ try
+ {
+ //adjust occurrence Attribut
+ if (setOcc)
+ {
+ oc = Integer.parseInt(e.getAttributeValue("occurrence"));
+ e.setAttribute("occurrence", Integer.toString(oc + 1));
+ }
+
+ //write morphilo-ObjectID in xml of corpmeta
+ if (setXlink)
+ {
+ Namespace xlinkNamespace = Namespace.getNamespace("xlink", "http://www.w3.org/1999/xlink");
+ MCRObject corpObj = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(getURLParameter(job, "objID")));
+ Document corpDoc = corpObj.createXML();
+ XPathExpression<Element> xpathEx = xpfac.compile("//corpuslink", Filters.element());
+ Element elm = xpathEx.evaluateFirst(corpDoc);
+ elm.setAttribute("href" , mcrIDString, xlinkNamespace);
+ }
+ mcrObj = new MCRObject(jdomDoc);
+ MCRMetadataManager.update(mcrObj);
+ QualityControl qc = new QualityControl(mcrObj);
+ }
+ catch(NumberFormatException except)
+ {
+ // ignore
+ }
+ }
+ }
+ if (!available) // if not available in datasets under the given conditions
+ {
+ lo.add(corpus.get(i));
+ }
+ }
+ return lo;
+ }
+
+As can be seen from the functionality of listing :ref:`getUnknowWords`, getting the unknown words of a corpus, is rather a side effect for the equally named method.
+More precisely, a Boolean (line 47) is set when the document is manipulated otherwise because it is clear that the word must exist then.
+If the Boolean remains false (line 77), the word is put on the list of words that have to be annotated manually. As already explained above, the
first loop runs through all words (corpus) and the following lines a solr result set is created. This set is also looped through and it is checked if the time range,
-the word type and the user are authorized. In the remainder, the occurrence attribute of the morphilo document can be incremented (setOcc is true) or/and the word is linked to the
-corpus meta data (setXlink is true). While all code lines are equivalent with
-what was explained in listing \ref{src:think}, it suffices to focus on an
+the word type and the user are authorized. In the remainder, the occurrence attribute of the morphilo document can be incremented (*setOcc* is true) or/and the word is linked to the
+corpus meta data (*setXlink* is true). While all code lines are equivalent with
+what was explained in listing :ref:`think`, it suffices to focus on an
additional name space, i.e.
-``xlink'' has to be defined (line \ref{ln:namespace}). Once the linking of word
-and corpus is set, the entire MyCoRe object has to be updated. This is done by the functionality of the framework (lines \ref{ln:updatestart}--\ref{ln:updateend}).
-At the end, an instance of \emph{QualityControl} is created.
-
-%QualityControl
-The class \emph{QualityControl} is instantiated with a constructor
-depicted in listing \ref{src:constructQC}.
-\begin{lstlisting}[language=java,caption={Constructor of QualityControl.java},label=src:constructQC,escapechar=|]
-private MCRObject mycoreObject;
-/* Constructor calls method to carry out quality control, i.e. if at least 20
- * different users agree 100% on the segments of the word under investigation
- */
-public QualityControl(MCRObject mycoreObject) throws Exception
-{
- this.mycoreObject = mycoreObject;
- if (getEqualObjectNumber() > 20)
- {
- addToMorphiloDB();
- }
-}
-\end{lstlisting}
+*xlink* has to be defined (line 60). Once the linking of word
+and corpus is set, the entire MyCoRe object has to be updated. This is done by the functionality of the framework (lines 67--69).
+At the end, an instance of *QualityControl* is created.
+
+The class *QualityControl* is instantiated with a constructor
+depicted in listing :ref:`constructQC`.
+
+.. _constructQC:
+
+.. code-block:: java
+ :caption: Constructor of QualityControl.java
+
+ private MCRObject mycoreObject;
+ /* Constructor calls method to carry out quality control, i.e. if at least 20
+ * different users agree 100% on the segments of the word under investigation
+ */
+ public QualityControl(MCRObject mycoreObject) throws Exception
+ {
+ this.mycoreObject = mycoreObject;
+ if (getEqualObjectNumber() > 20)
+ {
+ addToMorphiloDB();
+ }
+ }
+
The constructor takes an MyCoRe object, a potential word candidate for the
master data base, which is assigned to a private class variable because the
object is used though not changed by some other java methods.
-More importantly, there are two more methods: \emph{getEqualNumber()} and
-\emph{addToMorphiloDB()}. While the former initiates a process of counting and
+More importantly, there are two more methods: *getEqualNumber()* and
+*addToMorphiloDB()*. While the former initiates a process of counting and
comparing objects, the latter is concerned with calculating the correct number
-of occurrences from different, but not the same texts, and generating a MyCoRe object with the same content but with two different flags in the \emph{//service/servflags/servflag}-node, i.e. \emph{createdby='administrator'} and \emph{state='published'}.
-And of course, the \emph{occurrence} attribute is set to the newly calculated value. The logic corresponds exactly to what was explained in
-listing \ref{src:think} and will not be repeated here. The only difference are the paths compiled by the XPathFactory. They are
-\begin{itemize}
- \item[-] \emph{//service/servflags/servflag[@type='createdby']} and
- \item[-] \emph{//service/servstates/servstate[@classid='state']}.
-\end{itemize}
+of occurrences from different, but not the same texts, and generating a MyCoRe object with the same
+content but with two different flags in the *//service/servflags/servflag*-node, i.e. *createdby='administrator'* and *state='published'*.
+And of course, the *occurrence* attribute is set to the newly calculated value. The logic corresponds exactly to what was explained in
+listing :ref:`think` and will not be repeated here. The only difference are the paths compiled by the XPathFactory. They are
+
+* *//service/servflags/servflag[@type='createdby']* and
+* *//service/servstates/servstate[@classid='state']*.
+
It is more instructive to document how the number of occurrences is calculated. There are two steps involved. First, a list with all mycore objects that are
-equal to the object which the class is instantiated with (``mycoreObject'' in listing \ref{src:constructQC}) is created. This list is looped and all occurrence
+equal to the object which the class is instantiated with (*mycoreObject* in listing :ref:`constructQC`) is created. This list is looped and all occurrence
attributes are summed up. Second, all occurrences from equal texts are substracted. Equal texts are identified on the basis of its meta data and its derivate.
-There are some obvious shortcomings of this approach, which will be discussed in chapter \ref{chap:results}, section \ref{sec:improv}. Here, suffice it to
-understand the mode of operation. Listing \ref{src:equalOcc} shows a possible solution.
-\begin{lstlisting}[language=java,caption={Occurrence Extraction from Equal Texts (1)},label=src:equalOcc,escapechar=|]
-/* returns number of Occurrences if Objects are equal, zero otherwise
- */
-private int getOccurrencesFromEqualTexts(MCRObject mcrobj1, MCRObject mcrobj2) throws SAXException, IOException
-{
- int occurrences = 1;
- //extract corpmeta ObjectIDs from morphilo-Objects
- String crpID1 = getAttributeValue("//corpuslink", "href", mcrobj1);
- String crpID2 = getAttributeValue("//corpuslink", "href", mcrobj2);
- //get these two corpmeta Objects
- MCRObject corpo1 = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(crpID1));
- MCRObject corpo2 = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(crpID2));
- //are the texts equal? get list of 'processed-words' derivate
- String corp1DerivID = getAttributeValue("//structure/derobjects/derobject", "href", corpo1);
- String corp2DerivID = getAttributeValue("//structure/derobjects/derobject", "href", corpo2);
-
- ArrayList result = new ArrayList(getContentFromFile(corp1DerivID, ""));|\label{ln:writeContent}|
- result.remove(getContentFromFile(corp2DerivID, ""));|\label{ln:removeContent}|
- if (result.size() == 0) // the texts are equal
- {
- // extract occurrences of one the objects
- occurrences = Integer.parseInt(getAttributeValue("//morphiloContainer/morphilo/w", "occurrence", mcrobj1));
- }
- else
- {
- occurrences = 0; //project metadata happened to be the same, but texts are different
- }
- return occurrences;
-}
-\end{lstlisting}
-In this implementation, the ids from the \emph{corpmeta} data model are accessed via the xlink attribute in the morphilo documents.
-The method \emph{getAttributeValue(String, String, MCRObject)} does exactly the same as demonstrated earlier (see from line \ref{ln:namespace}
-on in listing \ref{src:getUnknowWords}). The underlying logic is that the texts are equal if exactly the same number of words were uploaded.
-So all words from one file are written to a list (line \ref{ln:writeContent}) and words from the other file are removed from the
-very same list (line \ref{ln:removeContent}). If this list is empty, then the exact same number of words must have been in both files and the occurrences
-are adjusted accordingly. Since this method is called from another private method that only contains a loop through all equal objects, one gets
-the occurrences from all equal texts. For reasons of confirmability, the looping method is also given:
-\begin{lstlisting}[language=java,caption={Occurrence Extraction from Equal Texts (2)},label=src:equalOcc2,escapechar=|]
-private int getOccurrencesFromEqualTexts() throws Exception
-{
- ArrayList<MCRObject> equalObjects = new ArrayList<MCRObject>();
- equalObjects = getAllEqualMCRObjects();
- int occurrences = 0;
- for (MCRObject obj : equalObjects)
- {
- occurrences = occurrences + getOccurrencesFromEqualTexts(mycoreObject, obj);
- }
- return occurrences;
-}
-\end{lstlisting}
-
-Now, the constructor in listing \ref{src:constructQC} reveals another method that rolls out an equally complex concatenation of procedures.
-As implied above, \emph{getEqualObjectNumber()} returns the number of equally annotated words. It does this by falling back to another
-method from which the size of the returned list is calculated (\emph{getAllEqualMCRObjects().size()}). Hence, we should care about
-\emph{getAllEqualMCRObjects()}. This method really has the same design as \emph{int getOccurrencesFromEqualTexts()} in listing \ref{src:equalOcc2}.
-The difference is that another method (\emph{Boolean compareMCRObjects(MCRObject, MCRObject, String)}) is used within the loop and
-that all equal objects are put into the list of MyCoRe objects that are returned. If this list comprises more than 20
-entries,\footnote{This number is somewhat arbitrary. It is inspired by the sample size n in t-distributed data.} the respective document
-will be integrated in the master data base by the process described above.
-The comparator logic is shown in listing \ref{src:compareMCR}.
-\begin{lstlisting}[language=java,caption={Comparison of MyCoRe objects},label=src:compareMCR,escapechar=|]
-private Boolean compareMCRObjects(MCRObject mcrobj1, MCRObject mcrobj2, String xpath) throws SAXException, IOException
-{
- Boolean isEqual = false;
- Boolean beginTime = false;
- Boolean endTime = false;
- Boolean occDiff = false;
- Boolean corpusDiff = false;
-
- String source = getXMLFromObject(mcrobj1, xpath);
- String target = getXMLFromObject(mcrobj2, xpath);
-
- XMLUnit.setIgnoreAttributeOrder(true);
- XMLUnit.setIgnoreComments(true);
- XMLUnit.setIgnoreDiffBetweenTextAndCDATA(true);
- XMLUnit.setIgnoreWhitespace(true);
- XMLUnit.setNormalizeWhitespace(true);
-
- //differences in occurrences, end, begin should be ignored
- try
- {
- Diff xmlDiff = new Diff(source, target);
- DetailedDiff dd = new DetailedDiff(xmlDiff);
- //counters for differences
- int i = 0;
- int j = 0;
- int k = 0;
- int l = 0;
- // list containing all differences
- List differences = dd.getAllDifferences();|\label{ln:difflist}|
- for (Object object : differences)
+
+.. code-block:: java
+ :caption: Occurrence Extraction from Equal Texts
+ :linenos:
+
+ /* returns number of Occurrences if Objects are equal, zero otherwise
+ */
+ private int getOccurrencesFromEqualTexts(MCRObject mcrobj1, MCRObject mcrobj2) throws SAXException, IOException
{
- Difference difference = (Difference) object;
- //@begin,@end,... node is not in the difference list if the count is 0
- if (difference.getControlNodeDetail().getXpathLocation().endsWith("@begin")) i++;|\label{ln:diffbegin}|
- if (difference.getControlNodeDetail().getXpathLocation().endsWith("@end")) j++;
- if (difference.getControlNodeDetail().getXpathLocation().endsWith("@occurrence")) k++;
- if (difference.getControlNodeDetail().getXpathLocation().endsWith("@corpus")) l++;|\label{ln:diffend}|
- //@begin and @end have different values: they must be checked if they fall right in the allowed time range
- if ( difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())
- && difference.getControlNodeDetail().getXpathLocation().endsWith("@begin")
- && (Integer.parseInt(difference.getControlNodeDetail().getValue()) < Integer.parseInt(difference.getTestNodeDetail().getValue())) )
+ int occurrences = 1;
+ //extract corpmeta ObjectIDs from morphilo-Objects
+ String crpID1 = getAttributeValue("//corpuslink", "href", mcrobj1);
+ String crpID2 = getAttributeValue("//corpuslink", "href", mcrobj2);
+ //get these two corpmeta Objects
+ MCRObject corpo1 = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(crpID1));
+ MCRObject corpo2 = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(crpID2));
+ //are the texts equal? get list of 'processed-words' derivate
+ String corp1DerivID = getAttributeValue("//structure/derobjects/derobject", "href", corpo1);
+ String corp2DerivID = getAttributeValue("//structure/derobjects/derobject", "href", corpo2);
+
+ ArrayList result = new ArrayList(getContentFromFile(corp1DerivID, ""));
+ result.remove(getContentFromFile(corp2DerivID, ""));
+ if (result.size() == 0) // the texts are equal
{
- beginTime = true;
+ // extract occurrences of one the objects
+ occurrences = Integer.parseInt(getAttributeValue("//morphiloContainer/morphilo/w", "occurrence", mcrobj1));
}
- if (difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())
- && difference.getControlNodeDetail().getXpathLocation().endsWith("@end")
- && (Integer.parseInt(difference.getControlNodeDetail().getValue()) > Integer.parseInt(difference.getTestNodeDetail().getValue())) )
+ else
{
- endTime = true;
+ occurrences = 0; //project metadata happened to be the same, but texts are different
}
- //attribute values of @occurrence and @corpus are ignored if they are different
- if (difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())
- && difference.getControlNodeDetail().getXpathLocation().endsWith("@occurrence"))
+ return occurrences;
+ }
+
+In this implementation, the ids from the *corpmeta* data model are accessed via the xlink attribute in the morphilo documents.
+The method *getAttributeValue(String, String, MCRObject)* does exactly the same as demonstrated earlier (see from line 60
+on in listing :ref:`getUnknowWords`). The underlying logic is that the texts are equal if exactly the same number of words were uploaded.
+So all words from one file are written to a list (line 16) and words from the other file are removed from the
+very same list (line 17). If this list is empty, then the exact same number of words must have been in both files and the occurrences
+are adjusted accordingly. Since this method is called from another private method that only contains a loop through all equal objects, one gets
+the occurrences from all equal texts. For reasons of confirmability, the looping method is also given:
+
+.. _equalOcc2:
+
+.. code-block:: java
+ :caption: Occurrence Extraction from Equal Texts (2)
+ :linenos:
+
+ private int getOccurrencesFromEqualTexts() throws Exception
+ {
+ ArrayList<MCRObject> equalObjects = new ArrayList<MCRObject>();
+ equalObjects = getAllEqualMCRObjects();
+ int occurrences = 0;
+ for (MCRObject obj : equalObjects)
{
- occDiff = true;
+ occurrences = occurrences + getOccurrencesFromEqualTexts(mycoreObject, obj);
}
- if (difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())
- && difference.getControlNodeDetail().getXpathLocation().endsWith("@corpus"))
- {
- corpusDiff = true;
- }
+ return occurrences;
}
- //if any of @begin, @end ... is identical set Boolean to true
- if (i == 0) beginTime = true;|\label{ln:zerobegin}|
- if (j == 0) endTime = true;
- if (k == 0) occDiff = true;
- if (l == 0) corpusDiff = true;|\label{ln:zeroend}|
- //if the size of differences is greater than the number of changes admitted in @begin, @end ... something else must be different
- if (beginTime && endTime && occDiff && corpusDiff && (i + j + k + l) == dd.getAllDifferences().size()) isEqual = true;|\label{ln:diffsum}|
- }
- catch (SAXException e)
- {
- e.printStackTrace();
- }
- catch (IOException e)
+
+
+Now, the constructor in listing :ref:`constructQC` reveals another method that rolls out an equally complex concatenation of procedures.
+As implied above, *getEqualObjectNumber()* returns the number of equally annotated words. It does this by falling back to another
+method from which the size of the returned list is calculated (*getAllEqualMCRObjects().size()*). Hence, we should care about
+*getAllEqualMCRObjects()*. This method really has the same design as *int getOccurrencesFromEqualTexts()* in listing :ref:`equalOcc2`.
+The difference is that another method (*Boolean compareMCRObjects(MCRObject, MCRObject, String)*) is used within the loop and
+that all equal objects are put into the list of MyCoRe objects that are returned. If this list comprises more than 20
+entries, [#f4]_ the respective document
+will be integrated in the master data base by the process described above.
+The comparator logic is shown in listing :ref:`compareMCR`.
+
+.. _compareMCR:
+
+.. code-block:: java
+ :caption: Comparison of MyCoRe objects
+ :linenos:
+ :emphasize-lines: 29, 34, 37, 64, 67, 69
+
+ private Boolean compareMCRObjects(MCRObject mcrobj1, MCRObject mcrobj2, String xpath) throws SAXException, IOException
{
- e.printStackTrace();
+ Boolean isEqual = false;
+ Boolean beginTime = false;
+ Boolean endTime = false;
+ Boolean occDiff = false;
+ Boolean corpusDiff = false;
+
+ String source = getXMLFromObject(mcrobj1, xpath);
+ String target = getXMLFromObject(mcrobj2, xpath);
+
+ XMLUnit.setIgnoreAttributeOrder(true);
+ XMLUnit.setIgnoreComments(true);
+ XMLUnit.setIgnoreDiffBetweenTextAndCDATA(true);
+ XMLUnit.setIgnoreWhitespace(true);
+ XMLUnit.setNormalizeWhitespace(true);
+
+ //differences in occurrences, end, begin should be ignored
+ try
+ {
+ Diff xmlDiff = new Diff(source, target);
+ DetailedDiff dd = new DetailedDiff(xmlDiff);
+ //counters for differences
+ int i = 0;
+ int j = 0;
+ int k = 0;
+ int l = 0;
+ // list containing all differences
+ List differences = dd.getAllDifferences();
+ for (Object object : differences)
+ {
+ Difference difference = (Difference) object;
+ //@begin,@end,... node is not in the difference list if the count is 0
+ if (difference.getControlNodeDetail().getXpathLocation().endsWith("@begin")) i++;
+ if (difference.getControlNodeDetail().getXpathLocation().endsWith("@end")) j++;
+ if (difference.getControlNodeDetail().getXpathLocation().endsWith("@occurrence")) k++;
+ if (difference.getControlNodeDetail().getXpathLocation().endsWith("@corpus")) l++;
+ //@begin and @end have different values: they must be checked if they fall right in the allowed time range
+ if ( difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())
+ && difference.getControlNodeDetail().getXpathLocation().endsWith("@begin")
+ && (Integer.parseInt(difference.getControlNodeDetail().getValue()) < Integer.parseInt(difference.getTestNodeDetail().getValue())) )
+ {
+ beginTime = true;
+ }
+ if (difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())
+ && difference.getControlNodeDetail().getXpathLocation().endsWith("@end")
+ && (Integer.parseInt(difference.getControlNodeDetail().getValue()) > Integer.parseInt(difference.getTestNodeDetail().getValue())) )
+ {
+ endTime = true;
+ }
+ //attribute values of @occurrence and @corpus are ignored if they are different
+ if (difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())
+ && difference.getControlNodeDetail().getXpathLocation().endsWith("@occurrence"))
+ {
+ occDiff = true;
+ }
+ if (difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())
+ && difference.getControlNodeDetail().getXpathLocation().endsWith("@corpus"))
+ {
+ corpusDiff = true;
+ }
+ }
+ //if any of @begin, @end ... is identical set Boolean to true
+ if (i == 0) beginTime = true;
+ if (j == 0) endTime = true;
+ if (k == 0) occDiff = true;
+ if (l == 0) corpusDiff = true;
+ //if the size of differences is greater than the number of changes admitted in @begin, @end ... something else must be different
+ if (beginTime && endTime && occDiff && corpusDiff && (i + j + k + l) == dd.getAllDifferences().size()) isEqual = true;
+ }
+ catch (SAXException e)
+ {
+ e.printStackTrace();
+ }
+ catch (IOException e)
+ {
+ e.printStackTrace();
+ }
+ return isEqual;
}
- return isEqual;
-}
-\end{lstlisting}
+
In this method, XMLUnit is heavily used to make all necessary node comparisons. The matter becomes more complicated, however, if some attributes
are not only ignored, but evaluated according to a given definition as it is the case for the time range. If the evaluator and builder classes are
not to be overwritten entirely because needed for evaluating other nodes of the
xml document, the above solution appears a bit awkward. So there is potential for improvement before the production version is to be programmed.
XMLUnit provides us with a
-list of the differences of the two documents (see line \ref{ln:difflist}). There are four differences allowed, that is, the attributes \emph{occurrence},
-\emph{corpus}, \emph{begin}, and \emph{end}. For each of them a Boolean variable is set. Because any of the attributes could also be equal to the master
+list of the differences of the two documents (see line 29). There are four differences allowed, that is, the attributes *occurrence*,
+*corpus*, *begin*, and *end*. For each of them a Boolean variable is set. Because any of the attributes could also be equal to the master
document and the difference list only contains the actual differences, one has to find a way to define both, equal and different, for the attributes.
This could be done by ignoring these nodes. Yet, this would not include testing if the beginning and ending dates fall into the range of the master
-document. Therefore the attributes are counted as lines \ref{ln:diffbegin} through \ref{ln:diffend} reveal. If any two documents
-differ in some of the four attributes just specified, then the sum of the counters (line \ref{ln:diffsum}) should not be greater than the collected differences
+document. Therefore the attributes are counted as lines 34 through 37 reveal. If any two documents
+differ in some of the four attributes just specified, then the sum of the counters (line 69) should not be greater than the collected differences
by XMLUnit. The rest of the if-tests assign truth values to the respective
Booleans. It is probably worth mentioning that if all counters are zero (lines
-\ref{ln:zerobegin}-\ref{ln:zeroend}) the attributes and values are identical and hence the Boolean has to be set explicitly. Otherwise the test in line \ref{ln:diffsum} would fail.
+64--67) the attributes and values are identical and hence the Boolean has to be set explicitly. Otherwise the test in line 69 would fail.
-%TagCorpusServlet
Once quality control (explained in detail further down) has been passed, it is
-the user's turn to interact further. By clicking on the option \emph{Manual tagging}, the \emph{TagCorpusServlet} will be callled. This servlet instantiates
-\emph{ProcessCorpusServlet} to get access to the \emph{getUnknownWords}-method, which delivers the words still to be
-processed and which overwrites the content of the file starting with \emph{untagged}. For the next word in \emph{leftovers} a new MyCoRe object is created
-using the JDOM API and added to the file beginning with \emph{processed}. In line \ref{ln:tagmanu} of listing \ref{src:tagservlet}, the previously defined
+the user's turn to interact further. By clicking on the option *Manual tagging*, the *TagCorpusServlet* will be callled. This servlet instantiates
+*ProcessCorpusServlet* to get access to the *getUnknownWords*-method, which delivers the words still to be
+processed and which overwrites the content of the file starting with *untagged*. For the next word in *leftovers* a new MyCoRe object is created
+using the JDOM API and added to the file beginning with *processed*. In line 16 of listing :ref:`tagservlet`, the previously defined
entry mask is called, with which the proposed word structure could be confirmed or changed. How the word structure is determined will be shown later in
the text.
-\begin{lstlisting}[language=java,caption={Manual Tagging Procedure},label=src:tagservlet,escapechar=|]
-...
-if (!leftovers.isEmpty())
-{
- ArrayList<String> processed = new ArrayList<String>();
- //processed.add(leftovers.get(0));
- JDOMorphilo jdm = new JDOMorphilo();
- MCRObject obj = jdm.createMorphiloObject(job, leftovers.get(0));|\label{ln:jdomobject}|
- //write word to be annotated in process list and save it
- Path filePathProc = pcs.getDerivateFilePath(job, "processed").getFileName();
- Path proc = root.resolve(filePathProc);
- processed = pcs.getContentFromFile(job, "processed");
- processed.add(leftovers.get(0));
- Files.write(proc, processed);
-
- //call entry mask for next word
- tagUrl = prop.getBaseURL() + "content/publish/morphilo.xed?id=" + obj.getId();|\label{ln:tagmanu}|
-}
-else
-{
- //initiate process to give a complete tagged file of the original corpus
- //if untagged-file is empty, match original file with morphilo
- //creator=administrator OR creator=username and write matches in a new file
- ArrayList<String> complete = new ArrayList<String>();
- ProcessCorpusServlet pcs2 = new ProcessCorpusServlet();
- complete = pcs2.getUnknownWords(
- pcs2.getContentFromFile(job, ""), //main corpus file
- pcs2.getCorpusMetadata(job, "def.datefrom"),
- pcs2.getCorpusMetadata(job, "def.dateuntil"),
- "", //wordtype
- false,
- false,
- true);
-
- Files.delete(p);
- MCRXMLFunctions mdm = new MCRXMLFunctions();
- String mainFile = mdm.getMainDocName(derivID);
- Path newRoot = root.resolve("tagged-" + mainFile);
- Files.write(newRoot, complete);
-
- //return to Menu page
- tagUrl = prop.getBaseURL() + "receive/" + corpID;
-}
-\end{lstlisting}
-At the point where no more items are in \emph{leftsovers} the \emph{getUnknownWords}-method is called whereas the last Boolean parameter
+
+.. _tagservlet:
+
+.. code-block:: java
+ :caption: Manual Tagging Procedure
+ :linenos:
+
+ ...
+ if (!leftovers.isEmpty())
+ {
+ ArrayList<String> processed = new ArrayList<String>();
+ //processed.add(leftovers.get(0));
+ JDOMorphilo jdm = new JDOMorphilo();
+ MCRObject obj = jdm.createMorphiloObject(job, leftovers.get(0));
+ //write word to be annotated in process list and save it
+ Path filePathProc = pcs.getDerivateFilePath(job, "processed").getFileName();
+ Path proc = root.resolve(filePathProc);
+ processed = pcs.getContentFromFile(job, "processed");
+ processed.add(leftovers.get(0));
+ Files.write(proc, processed);
+
+ //call entry mask for next word
+ tagUrl = prop.getBaseURL() + "content/publish/morphilo.xed?id=" + obj.getId();
+ }
+ else
+ {
+ //initiate process to give a complete tagged file of the original corpus
+ //if untagged-file is empty, match original file with morphilo
+ //creator=administrator OR creator=username and write matches in a new file
+ ArrayList<String> complete = new ArrayList<String>();
+ ProcessCorpusServlet pcs2 = new ProcessCorpusServlet();
+ complete = pcs2.getUnknownWords(
+ pcs2.getContentFromFile(job, ""), //main corpus file
+ pcs2.getCorpusMetadata(job, "def.datefrom"),
+ pcs2.getCorpusMetadata(job, "def.dateuntil"),
+ "", //wordtype
+ false,
+ false,
+ true);
+
+ Files.delete(p);
+ MCRXMLFunctions mdm = new MCRXMLFunctions();
+ String mainFile = mdm.getMainDocName(derivID);
+ Path newRoot = root.resolve("tagged-" + mainFile);
+ Files.write(newRoot, complete);
+
+ //return to Menu page
+ tagUrl = prop.getBaseURL() + "receive/" + corpID;
+ }
+
+At the point where no more items are in *leftsovers* the *getUnknownWords*-method is called whereas the last Boolean parameter
is set true. This indicates that the array list containing all available and relevant data to the respective user is returned as seen in
the code snippet in listing \ref{src:writeAll}.
-\begin{lstlisting}[language=java,caption={Code snippet to deliver all data to the user},label=src:writeAll,escapechar=|]
-...
-// all data is written to lo in TEI
-if (writeAllData && isAuthorized && timeCorrect)
-{
- XPathExpression<Element> xpath = xpfac.compile("//morphiloContainer/morphilo", Filters.element());
- for (Element e : xpath.evaluate(jdomDoc))
- {
- XMLOutputter outputter = new XMLOutputter();
- outputter.setFormat(Format.getPrettyFormat());
- lo.add(outputter.outputString(e.getContent()));
- }
-}
-...
-\end{lstlisting}
-The complete list (\emph{lo}) is written to yet a third file starting with \emph{tagged} and finally returned to the main project webpage.
-
-%JDOMorphilo
+
+.. code-block:: java
+ :caption: Code snippet to deliver all data to the user
+
+ ...
+ // all data is written to lo in TEI
+ if (writeAllData && isAuthorized && timeCorrect)
+ {
+ XPathExpression<Element> xpath = xpfac.compile("//morphiloContainer/morphilo", Filters.element());
+ for (Element e : xpath.evaluate(jdomDoc))
+ {
+ XMLOutputter outputter = new XMLOutputter();
+ outputter.setFormat(Format.getPrettyFormat());
+ lo.add(outputter.outputString(e.getContent()));
+ }
+ }
+ ...
+
+The complete list (*lo*) is written to yet a third file starting with *tagged* and finally returned to the main project webpage.
+
The interesting question is now where does the word structure come from, which is filled in the entry mask as asserted above.
-In listing \ref{src:tagservlet} line \ref{ln:jdomobject}, one can see that a JDOM object is created and the method
-\emph{createMorphiloObject(MCRServletJob, String)} is called. The string parameter is the word that needs to be analyzed.
-Most of the method is a mere application of the JDOM API given the data model in chapter \ref{chap:concept} section
-\ref{subsec:datamodel} and listing \ref{lst:worddatamodel}. That means namespaces, elements and their attributes are defined in the correct
+In listing :ref:`tagservlet` line 7, one can see that a JDOM object is created and the method
+*createMorphiloObject(MCRServletJob, String)* is called. The string parameter is the word that needs to be analyzed.
+Most of the method is a mere application of the JDOM API given the data model in :ref:`concept` and listing :ref:`worddatamodel`. That means namespaces, elements and their attributes are defined in the correct
order and hierarchy.
To fill the elements and attributes with text, i.e. prefixes, suffixes, stems, etc., a Hashmap -- containing the morpheme as
@@ -594,256 +631,292 @@ key and its position as value -- are created that are filled with the results fr
or suffixes respectively are put in the hashmap, the same number of xml elements are created. As a final step, a valid MyCoRe id is generated using
the existing MyCoRe functionality, the object is created and returned to the TagCorpusServlet.
-%AffixStripper explanation
Last, the analyses of the word structure will be considered. It is implemented
-in the \emph{AffixStripper.java} file.
+in the *AffixStripper.java* file.
All lexical affix morphemes and their allomorphs as well as the inflections were extracted from the
-OED\footnote{Oxford English Dictionary http://www.oed.com/} and saved as enumerated lists (see the example in listing \ref{src:enumPref}).
+`Oxford English Dictionary <http://www.oed.com/>`_ and saved as enumerated lists (see the example in listing :ref:`enumPref`).
The allomorphic items of these lists are mapped successively to the beginning in the case of prefixes
-(see listing \ref{src:analyzePref}, line \ref{ln:prefLoop}) or to the end of words in the case of suffixes
-(see listing \ref{src:analyzeSuf}). Since each
+(see listing :ref:`analyzePref`, line 7) or to the end of words in the case of suffixes
+(see listing :ref:`analyzeSuf`). Since each
morphemic variant maps to its morpheme right away, it makes sense to use the morpheme and so
implicitly keep the relation to its allomorph.
-\begin{lstlisting}[language=java,caption={Enumeration Example for the Prefix "over"},label=src:enumPref,escapechar=|]
-package custom.mycore.addons.morphilo;
-
-public enum PrefixEnum {
-...
- over("over"), ufer("over"), ufor("over"), uferr("over"), uvver("over"), obaer("over"), ober("over)"), ofaer("over"),
- ofere("over"), ofir("over"), ofor("over"), ofer("over"), ouer("over"),oferr("over"), offerr("over"), offr("over"), aure("over"),
- war("over"), euer("over"), oferre("over"), oouer("over"), oger("over"), ouere("over"), ouir("over"), ouire("over"),
- ouur("over"), ouver("over"), ouyr("over"), ovar("over"), overe("over"), ovre("over"),ovur("over"), owuere("over"), owver("over"),
- houyr("over"), ouyre("over"), ovir("over"), ovyr("over"), hover("over"), auver("over"), awver("over"), ovver("over"),
- hauver("over"), ova("over"), ove("over"), obuh("over"), ovah("over"), ovuh("over"), ofowr("over"), ouuer("over"), oure("over"),
- owere("over"), owr("over"), owre("over"), owur("over"), owyr("over"), our("over"), ower("over"), oher("over"),
- ooer("over"), oor("over"), owwer("over"), ovr("over"), owir("over"), oar("over"), aur("over"), oer("over"), ufara("over"),
- ufera("over"), ufere("over"), uferra("over"), ufora("over"), ufore("over"), ufra("over"), ufre("over"), ufyrra("over"),
- yfera("over"), yfere("over"), yferra("over"), uuera("over"), ufe("over"), uferre("over"), uuer("over"), uuere("over"),
- vfere("over"), vuer("over"), vuere("over"), vver("over"), uvvor("over") ...
-...chap:results
- private String morpheme;
- //constructor
- PrefixEnum(String morpheme)
- {
- this.morpheme = morpheme;
- }
- //getter Method
-
- public String getMorpheme()
- {
- return this.morpheme;
- }
-}
-\end{lstlisting}
-As can be seen in line \ref{ln:prefPutMorph} in listing \ref{src:analyzePref}, the morpheme is saved to a hash map together with its position, i.e. the size of the
-map plus one at the time being. In line \ref{ln:prefCutoff} the \emph{analyzePrefix} method is recursively called until no more matches can be made.
-
-\begin{lstlisting}[language=java,caption={Method to recognize prefixes},label=src:analyzePref,escapechar=|]
-private Map<String, Integer> prefixMorpheme = new HashMap<String,Integer>();
-...
-private void analyzePrefix(String restword)
-{
- if (!restword.isEmpty()) //Abbruchbedingung fuer Rekursion
- {
- for (PrefixEnum prefEnum : PrefixEnum.values())|\label{ln:prefLoop}|
- {
- String s = prefEnum.toString();
- if (restword.startsWith(s))
+.. _enumPref:
+
+.. code-block:: java
+ :caption: Enumeration Example for the Prefix *over*
+
+ package custom.mycore.addons.morphilo;
+ public enum PrefixEnum {
+ ...
+ over("over"), ufer("over"), ufor("over"), uferr("over"), uvver("over"), obaer("over"), ober("over)"), ofaer("over"),
+ ofere("over"), ofir("over"), ofor("over"), ofer("over"), ouer("over"),oferr("over"), offerr("over"), offr("over"), aure("over"),
+ war("over"), euer("over"), oferre("over"), oouer("over"), oger("over"), ouere("over"), ouir("over"), ouire("over"),
+ ouur("over"), ouver("over"), ouyr("over"), ovar("over"), overe("over"), ovre("over"),ovur("over"), owuere("over"), owver("over"),
+ houyr("over"), ouyre("over"), ovir("over"), ovyr("over"), hover("over"), auver("over"), awver("over"), ovver("over"),
+ hauver("over"), ova("over"), ove("over"), obuh("over"), ovah("over"), ovuh("over"), ofowr("over"), ouuer("over"), oure("over"),
+ owere("over"), owr("over"), owre("over"), owur("over"), owyr("over"), our("over"), ower("over"), oher("over"),
+ ooer("over"), oor("over"), owwer("over"), ovr("over"), owir("over"), oar("over"), aur("over"), oer("over"), ufara("over"),
+ ufera("over"), ufere("over"), uferra("over"), ufora("over"), ufore("over"), ufra("over"), ufre("over"), ufyrra("over"),
+ yfera("over"), yfere("over"), yferra("over"), uuera("over"), ufe("over"), uferre("over"), uuer("over"), uuere("over"),
+ vfere("over"), vuer("over"), vuere("over"), vver("over"), uvvor("over") ...
+ private String morpheme;
+ //constructor
+ PrefixEnum(String morpheme)
{
- prefixMorpheme.put(s, prefixMorpheme.size() + 1);|\label{ln:prefPutMorph}|
- //cut off the prefix that is added to the list
- analyzePrefix(restword.substring(s.length()));|\label{ln:prefCutoff}|
+ this.morpheme = morpheme;
}
- else
+ //getter Method
+
+ public String getMorpheme()
{
- analyzePrefix("");
+ return this.morpheme;
+ }
+ }
+
+As can be seen in line 12 in listing :ref:`analyzePref`, the morpheme is saved to a hash map together with its position, i.e. the size of the
+map plus one at the time being. In line 14 the *analyzePrefix* method is recursively called until no more matches can be made.
+
+.. _analyzePref:
+
+.. code-block:: java
+ :caption: Method to recognize prefixes
+ :linenos:
+ :emphasize-lines: 12
+
+ private Map<String, Integer> prefixMorpheme = new HashMap<String,Integer>();
+ ...
+ private void analyzePrefix(String restword)
+ {
+ if (!restword.isEmpty()) //Abbruchbedingung fuer Rekursion
+ {
+ for (PrefixEnum prefEnum : PrefixEnum.values())
+ {
+ String s = prefEnum.toString();
+ if (restword.startsWith(s))
+ {
+ prefixMorpheme.put(s, prefixMorpheme.size() + 1);
+ //cut off the prefix that is added to the list
+ analyzePrefix(restword.substring(s.length()));
+ }
+ else
+ {
+ analyzePrefix("");
+ }
+ }
}
}
- }
-}
-\end{lstlisting}
The recognition of suffixes differs only in the cut-off direction since suffixes occur at the end of a word.
-Hence, line \ref{ln:prefCutoff} in listing \ref{src:analyzePref} reads in the case of suffixes.
+Hence, line 14 in listing :ref:`analyzePref` reads in the case of suffixes.
-\begin{lstlisting}[language=java,caption={Cut-off mechanism for suffixes},label=src:analyzeSuf,escapechar=|]
-analyzeSuffix(restword.substring(0, restword.length() - s.length()));
-\end{lstlisting}
+.. _analyzeSuf:
+
+.. code-block:: java
+ :caption: Cut-off mechanism for suffixes
+
+ analyzeSuffix(restword.substring(0, restword.length() - s.length()));
It is important to note that inflections are suffixes (in the given model case of Middle English morphology) that usually occur at the very
end of a word, i.e. after all lexical suffixes, only once. It follows that inflections
have to be recognized at first without any repetition. So the procedure for inflections can be simplified
-to a substantial degree as listing \ref{src:analyzeInfl} shows.
-
-\begin{lstlisting}[language=java,caption={Method to recognize inflections},label=src:analyzeInfl,escapechar=|]
-private String analyzeInflection(String wrd)
-{
- String infl = "";
- for (InflectionEnum inflEnum : InflectionEnum.values())
- {
- if (wrd.endsWith(inflEnum.toString()))
+to a substantial degree as listing :ref:`analyzeInfl` shows.
+
+.. _analyzeInfl:
+
+.. code-block:: java
+ :caption: Method to recognize inflections
+
+ private String analyzeInflection(String wrd)
{
- infl = inflEnum.toString();
+ String infl = "";
+ for (InflectionEnum inflEnum : InflectionEnum.values())
+ {
+ if (wrd.endsWith(inflEnum.toString()))
+ {
+ infl = inflEnum.toString();
+ }
+ }
+ return infl;
}
- }
- return infl;
-}
-\end{lstlisting}
Unfortunately the embeddedness problem prevents a very simple algorithm. Embeddedness occurs when a lexical item
-is a substring of another lexical item. To illustrate, the suffix \emph{ion} is also contained in the suffix \emph{ation}, as is
-\emph{ent} in \emph{ment}, and so on. The embeddedness problem cannot be solved completely on the basis of linear modelling, but
+is a substring of another lexical item. To illustrate, the suffix *ion* is also contained in the suffix *ation*, as is
+*ent* in *ment*, and so on. The embeddedness problem cannot be solved completely on the basis of linear modelling, but
for a large part of embedded items one can work around it using implicitly Zipf's law, i.e. the correlation between frequency
and length of lexical items. The longer a word becomes, the less frequent it will occur. The simplest logic out of it is to assume
that longer suffixes (measured in letters) are preferred over shorter suffixes because it is more likely tha the longer the suffix string becomes,
-the more likely it is one (as opposed to several) suffix unit(s). This is done in listing \ref{src:embedAffix}, whereas
-the inner class \emph{sortedByLengthMap} returns a list sorted by length and the loop from line \ref{ln:deleteAffix} onwards deletes
+the more likely it is one (as opposed to several) suffix unit(s). This is done in listing :ref:`embedAffix`, whereas
+the inner class *sortedByLengthMap* returns a list sorted by length and the loop from line 17 onwards deletes
the respective substrings.
-\begin{lstlisting}[language=java,caption={Method to workaround embeddedness},label=src:embedAffix,escapechar=|]
-private Map<String, Integer> sortOutAffixes(Map<String, Integer> affix)
-{
- Map<String,Integer> sortedByLengthMap = new TreeMap<String, Integer>(new Comparator<String>()
+.. _embedAffix:
+
+.. code-block:: java
+ :caption: Method to workaround embeddedness
+ :linenos:
+
+ private Map<String, Integer> sortOutAffixes(Map<String, Integer> affix)
{
- @Override
- public int compare(String s1, String s2)
- {
- int cmp = Integer.compare(s1.length(), s2.length());
- return cmp != 0 ? cmp : s1.compareTo(s2);
- }
- }
- );
- sortedByLengthMap.putAll(affix);
- ArrayList<String> al1 = new ArrayList<String>(sortedByLengthMap.keySet());
- ArrayList<String> al2 = al1;
- Collections.reverse(al2);
- for (String s2 : al1)|\label{ln:deleteAffix}|
- {
- for (String s1 : al2)
- if (s1.contains(s2) && s1.length() > s2.length())
+ Map<String,Integer> sortedByLengthMap = new TreeMap<String, Integer>(new Comparator<String>()
+ {
+ @Override
+ public int compare(String s1, String s2)
+ {
+ int cmp = Integer.compare(s1.length(), s2.length());
+ return cmp != 0 ? cmp : s1.compareTo(s2);
+ }
+ }
+ );
+ sortedByLengthMap.putAll(affix);
+ ArrayList<String> al1 = new ArrayList<String>(sortedByLengthMap.keySet());
+ ArrayList<String> al2 = al1;
+ Collections.reverse(al2);
+ for (String s2 : al1)
{
- affix.remove(s2);
- }
+ for (String s1 : al2)
+ if (s1.contains(s2) && s1.length() > s2.length())
+ {
+ affix.remove(s2);
+ }
+ }
+ return affix;
}
- return affix;
-}
-\end{lstlisting}
-
-Finally, the position of the affix has to be calculated because the hashmap in line \ref{ln:prefPutMorph} in
-listing \ref{src:analyzePref} does not keep the original order for changes taken place in addressing the affix embeddedness
-(listing \ref{src:embedAffix}). Listing \ref{src:affixPos} depicts the preferred solution.
-The recursive construction of the method is similar to \emph{private void analyzePrefix(String)} (listing \ref{src:analyzePref})
-only that the two affix types are handled in one method. For that, an additional parameter taking the form either \emph{suffix}
-or \emph{prefix} is included.
-
-\begin{lstlisting}[language=java,caption={Method to determine position of the affix},label=src:affixPos,escapechar=|]
-private void getAffixPosition(Map<String, Integer> affix, String restword, int pos, String affixtype)
-{
- if (!restword.isEmpty()) //Abbruchbedingung fuer Rekursion
- {
- for (String s : affix.keySet())
+
+Finally, the position of the affix has to be calculated because the hashmap in line 12 in
+listing :ref:`analyzePref` does not keep the original order for changes taken place in addressing the affix embeddedness
+(listing :ref:`embedAffix`). Listing :ref:`affixPos` depicts the preferred solution.
+The recursive construction of the method is similar to *private void analyzePrefix(String)* (listing :ref:`analyzePref`)
+only that the two affix types are handled in one method. For that, an additional parameter taking the form either *suffix*
+or *prefix* is included.
+
+.. _affixPos:
+
+.. code-block:: java
+ :caption: Method to determine position of the affix
+
+ private void getAffixPosition(Map<String, Integer> affix, String restword, int pos, String affixtype)
{
- if (restword.startsWith(s) && affixtype.equals("prefix"))
- {
- pos++;
- prefixMorpheme.put(s, pos);
- //prefixAllomorph.add(pos-1, restword.substring(s.length()));
- getAffixPosition(affix, restword.substring(s.length()), pos, affixtype);
- }
- else if (restword.endsWith(s) && affixtype.equals("suffix"))
- {
- pos++;
- suffixMorpheme.put(s, pos);
- //suffixAllomorph.add(pos-1, restword.substring(s.length()));
- getAffixPosition(affix, restword.substring(0, restword.length() - s.length()), pos, affixtype);
- }
- else
+ if (!restword.isEmpty()) //Abbruchbedingung fuer Rekursion
{
- getAffixPosition(affix, "", pos, affixtype);
+ for (String s : affix.keySet())
+ {
+ if (restword.startsWith(s) && affixtype.equals("prefix"))
+ {
+ pos++;
+ prefixMorpheme.put(s, pos);
+ //prefixAllomorph.add(pos-1, restword.substring(s.length()));
+ getAffixPosition(affix, restword.substring(s.length()), pos, affixtype);
+ }
+ else if (restword.endsWith(s) && affixtype.equals("suffix"))
+ {
+ pos++;
+ suffixMorpheme.put(s, pos);
+ //suffixAllomorph.add(pos-1, restword.substring(s.length()));
+ getAffixPosition(affix, restword.substring(0, restword.length() - s.length()), pos, affixtype);
+ }
+ else
+ {
+ getAffixPosition(affix, "", pos, affixtype);
+ }
+ }
}
}
- }
-}
-\end{lstlisting}
-To give the complete word structure, the root of a word should also be provided. In listing \ref{src:rootAnalyze} a simple solution is offered, however,
+To give the complete word structure, the root of a word should also be provided. In listing :ref:`rootAnalyze` a simple solution is offered, however,
considering compounds as words consisting of more than one root.
-\begin{lstlisting}[language=java,caption={Method to determine roots},label=src:rootAnalyze,escapechar=|]
-private ArrayList<String> analyzeRoot(Map<String, Integer> pref, Map<String, Integer> suf, int stemNumber)
-{
- ArrayList<String> root = new ArrayList<String>();
- int j = 1; //one root always exists
- // if word is a compound several roots exist
- while (j <= stemNumber)
- {
- j++;
- String rest = lemma;|\label{ln:lemma}|
-
- for (int i=0;i<pref.size();i++)
+
+.. _rootAnalyze:
+
+.. code-block:: java
+ :caption: Method to determine roots
+
+ private ArrayList<String> analyzeRoot(Map<String, Integer> pref, Map<String, Integer> suf, int stemNumber)
{
- for (String s : pref.keySet())
+ ArrayList<String> root = new ArrayList<String>();
+ int j = 1; //one root always exists
+ // if word is a compound several roots exist
+ while (j <= stemNumber)
{
- //if (i == pref.get(s))
- if (rest.length() > s.length() && s.equals(rest.substring(0, s.length())))
- {
- rest = rest.substring(s.length(),rest.length());
+ j++;
+ String rest = lemma;
+
+ for (int i=0;i<pref.size();i++)
+ {
+ for (String s : pref.keySet())
+ {
+ //if (i == pref.get(s))
+ if (rest.length() > s.length() && s.equals(rest.substring(0, s.length())))
+ {
+ rest = rest.substring(s.length(),rest.length());
+ }
+ }
}
+
+ for (int i=0;i<suf.size();i++)
+ {
+ for (String s : suf.keySet())
+ {
+ //if (i == suf.get(s))
+ if (s.length() < rest.length() && (s.equals(rest.substring(rest.length() - s.length(), rest.length()))))
+ {
+ rest = rest.substring(0, rest.length() - s.length());
+ }
+ }
+ }
+ root.add(rest);
}
+ return root;
}
-
- for (int i=0;i<suf.size();i++)
- {
- for (String s : suf.keySet())
- {
- //if (i == suf.get(s))
- if (s.length() < rest.length() && (s.equals(rest.substring(rest.length() - s.length(), rest.length()))))
- {
- rest = rest.substring(0, rest.length() - s.length());
- }
- }
- }
- root.add(rest);
- }
- return root;
-}
-\end{lstlisting}
+
The logic behind this method is that the root is the remainder of a word when all prefixes and suffixes are substracted.
So the loops run through the number of prefixes and suffixes at each position and substract the affix. Really, there is
some code doubling with the previously described methods, which could be eliminated by making it more modular in a possible
-refactoring phase. Again, this is not the concern of a prototype. Line \ref{ln:lemma} defines the initial state of a root,
-which is the case for monomorphemic words. The \emph{lemma} is defined as the wordtoken without the inflection. Thus listing
-\ref{src:lemmaAnalyze} reveals how the class variable is calculated
-\begin{lstlisting}[language=java,caption={Method to determine lemma},label=src:lemmaAnalyze,escapechar=|]
-/*
- * Simplification: lemma = wordtoken - inflection
- */
-private String analyzeLemma(String wrd, String infl)
-{
- return wrd.substring(0, wrd.length() - infl.length());
-}
-\end{lstlisting}
-The constructor of \emph{AffixStripper} calls the method \emph{analyzeWord()}
+refactoring phase. Again, this is not the concern of a prototype. Line 9 defines the initial state of a root,
+which is the case for monomorphemic words. The *lemma* is defined as the wordtoken without the inflection. Thus listing
+:ref:`lemmaAnalyze` reveals how the class variable is calculated
+
+.. _lemmaAnalyze:
+
+.. code-block:: java
+ :caption: Method to determine lemma
+
+ /*
+ * Simplification: lemma = wordtoken - inflection
+ */
+ private String analyzeLemma(String wrd, String infl)
+ {
+ return wrd.substring(0, wrd.length() - infl.length());
+ }
+
+The constructor of *AffixStripper* calls the method *analyzeWord()*
whose only job is to calculate each structure element in the correct order
-(listing \ref{src:lemmaAnalyze}). All structure elements are also provided by getters.
-\begin{lstlisting}[language=java,caption={Method to determine all word structure},label=src:lemmaAnalyze,escapechar=|]
-private void analyzeWord()
-{
- //analyze inflection first because it always occurs at the end of a word
- inflection = analyzeInflection(wordtoken);
- lemma = analyzeLemma(wordtoken, inflection);
- analyzePrefix(lemma);
- analyzeSuffix(lemma);
- getAffixPosition(sortOutAffixes(prefixMorpheme), lemma, 0, "prefix");
- getAffixPosition(sortOutAffixes(suffixMorpheme), lemma, 0, "suffix");
- prefixNumber = prefixMorpheme.size();
- suffixNumber = suffixMorpheme.size();
- wordroot = analyzeRoot(prefixMorpheme, suffixMorpheme, getStemNumber());
-}
-\end{lstlisting}
+(listing :ref:`lemmaAnalyze`. All structure elements are also provided by getters.
+
+
+.. code-block:: java
+ :caption: Method to determine all word structure
+
+ private void analyzeWord()
+ {
+ //analyze inflection first because it always occurs at the end of a word
+ inflection = analyzeInflection(wordtoken);
+ lemma = analyzeLemma(wordtoken, inflection);
+ analyzePrefix(lemma);
+ analyzeSuffix(lemma);
+ getAffixPosition(sortOutAffixes(prefixMorpheme), lemma, 0, "prefix");
+ getAffixPosition(sortOutAffixes(suffixMorpheme), lemma, 0, "suffix");
+ prefixNumber = prefixMorpheme.size();
+ suffixNumber = suffixMorpheme.size();
+ wordroot = analyzeRoot(prefixMorpheme, suffixMorpheme, getStemNumber());
+ }
To conclude, the Morphilo implementation as presented here, aims at fulfilling the task of a working prototype. It is important to note
that it neither claims to be a very efficient nor a ready software program to be used in production. However, it marks a crucial milestone
on the way to a production system. At some listings sources of improvement were made explicit; at others no suggestions were made. In the latter
case this does not imply that there is no potential for improvement. Once acceptability tests are carried out, it will be the task of a follow up project
-to identify these potentials and implement them accordingly.
\ No newline at end of file
+to identify these potentials and implement them accordingly.
+
+.. rubric:: Notes
+
+.. [#f4] This number is somewhat arbitrary. It is inspired by the sample size n in t-distributed data.
diff --git a/Morphilo_doc/_build/html/_sources/source/datamodel.rst.txt b/Morphilo_doc/_build/html/_sources/source/datamodel.rst.txt
index f206ef3ffb8967f600e499b5b76eee996ac7de31..6274276ba44db0cc5450b28bc208289b939610d5 100644
--- a/Morphilo_doc/_build/html/_sources/source/datamodel.rst.txt
+++ b/Morphilo_doc/_build/html/_sources/source/datamodel.rst.txt
@@ -1,6 +1,8 @@
Data Model
==========
+.. _concept:
+
Conceptualization
-----------------
@@ -11,24 +13,24 @@ and multi-user processing is necessary. In addition, the framework should
support web technologies, be well documented, and easy to extent. Ideally, the
MVC pattern is realized.
-\subsection{Data Model}\label{subsec:datamodel}
-The guidelines of the
-\emph{TEI}-standard\footnote{http://www.tei-c.org/release/doc/tei-p5-doc/en/Guidelines.pdf} on the
-word level are defined in line with the structure defined above in section \ref{subsec:morphologicalSystems}.
-In listing \ref{lst:teiExamp} an
+The guidelines of the `TEI standard <http://www.tei-c.org/release/doc/tei-p5-doc/en/Guidelines.pdf>`_ on the
+word level are defined in line with the defined word structure.
+In listing :ref:`teiexamp` an
example is given for a possible markup at the word level for
-\emph{comfortable}.\footnote{http://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-m.html}
-
-\begin{lstlisting}[language=XML,
-caption={TEI-example for 'comfortable'},label=lst:teiExamp]
-<w type="adjective">
- <m type="base">
- <m type="prefix" baseForm="con">com</m>
- <m type="root">fort</m>
- </m>
- <m type="suffix">able</m>
-</w>
-\end{lstlisting}
+`comfortable <http://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-m.html>`_
+
+.. _teiexamp:
+
+.. code-block:: xml
+ :caption: TEI-example for *comfortable*
+
+ <w type="adjective">
+ <m type="base">
+ <m type="prefix" baseForm="con">com</m>
+ <m type="root">fort</m>
+ </m>
+ <m type="suffix">able</m>
+ </w>
This data model reflects just one theoretical conception of a word structure model.
Crucially, the model emanates from the assumption
@@ -38,7 +40,7 @@ other hand, is enclosed in the base, which basically means a stronger lexical,
and less abstract, attachment to the root of a word. Modeling prefixes and suffixes on different
hierarchical levels has important consequences for the branching direction at
subword level (here right-branching). Left the theoretical interest aside, the
-choice of the TEI standard is reasonable with view to a sustainable architecture that allows for
+choice of the *TEI*-standard is reasonable with view to a sustainable architecture that allows for
exchanging data with little to no additional adjustments.
The negative account is that the model is not eligible for all languages.
@@ -51,191 +53,197 @@ stem and corresponds to the overwhelming majority of all research carried out
Implementation
--------------
-As laid out in the task analysis in section \ref{subsec:datamodel}, it is
-advantageous to use established standards. It was also shown that it makes sense
+It is
+advantageous to use established standardsn and it makes sense
to keep the meta data of each corpus separate from the data model used for the
words to be analyzed.
-For the present case, the TEI-standard was identified as an
+For the present case, the *TEI*-standard was identified as an
appropriate markup for words. In terms of the implementation this means that
-the TEI guidelines have to be implemented as an object type compatible with the chosen
+the *TEI*-guidelines have to be implemented as an object type compatible with the chosen
repository framework. However, the TEI standard is not complete regarding the
diachronic dimension, i.e. information on the development of the word. To
be compatible with the elements of the TEI standard on the one hand
and to best meet the requirements of the application on the other hand, some attributes
are added. This solution allows for processing the xml files according to
-the TEI standard by ignoring the additional attributes and at the same
+the *TEI*-standard by ignoring the additional attributes and at the same
time, if needed, additional markup can be extracted. The additional attributes
-comprise a link to the corpus meta data, but also \emph{position} and
-\emph{occurrence} of the affixes.
+comprise a link to the corpus meta data, but also *position* and
+*occurrence* of the affixes.
Information on the position and some quantification thereof are potentially relevant for a
wealth of research questions, such as predictions on the productivity of
derivatives and their interaction with the phonological or syntactic modules. So they were included
with respect to future use.
For reasons of efficiency in subsequent processing,
-the historic dates \emph{begin} and \emph{end} were included in both the word
+the historic dates *begin* and *end* were included in both the word
data model and the corpus data model. The result of the word data model is given
-in listing \ref{lst:worddatamodel}.
+in listing :ref:`worddatamodel`.
Whereas attributes of the objecttype are specific to the repository framework, the TEI structure can be
recognized in the hierarchy of the meta data element starting with the name
-\emph{w} (line \ref{src:wordbegin}).
-
-\begin{lstlisting}[language=XML,caption={Word Data
-model},label=lst:worddatamodel,escapechar=|] <?xml version="1.0" encoding="UTF-8"?>
-<objecttype
- name="morphilo"
- isChild="true"
- isParent="true"
- hasDerivates="true"
- xmlns:xs="http://www.w3.org/2001/XMLSchema"
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:noNamespaceSchemaLocation="datamodel.xsd">
- <metadata>
- <element name="morphiloContainer" type="xml" style="dontknow"
- notinherit="true" heritable="false">
- <xs:sequence>
- <xs:element name="morphilo">
- <xs:complexType>
- <xs:sequence>
- <xs:element name="w" minOccurs="0" maxOccurs="unbounded">|label{src:wordbegin}|
- <xs:complexType mixed="true">
- <xs:sequence>
- <!-- stem -->
- <xs:element name="m1" minOccurs="0" maxOccurs="unbounded">
- <xs:complexType mixed="true">
- <xs:sequence>
- <!-- base -->
- <xs:element name="m2" minOccurs="0" maxOccurs="unbounded">
- <xs:complexType mixed="true">
- <xs:sequence>
- <!-- root -->
- <xs:element name="m3" minOccurs="0" maxOccurs="unbounded">
- <xs:complexType mixed="true">
- <xs:attribute name="type" type="xs:string"/>
- </xs:complexType>
- </xs:element>
- <!-- prefix -->
- <xs:element name="m4" minOccurs="0" maxOccurs="unbounded">
- <xs:complexType mixed="true">
- <xs:attribute name="type" type="xs:string"/>
- <xs:attribute name="PrefixbaseForm" type="xs:string"/>
- <xs:attribute name="position" type="xs:string"/>
- </xs:complexType>
- </xs:element>
- </xs:sequence>
- <xs:attribute name="type" type="xs:string"/>
- </xs:complexType>
- </xs:element>
- <!-- suffix -->
- <xs:element name="m5" minOccurs="0" maxOccurs="unbounded">
- <xs:complexType mixed="true">
- <xs:attribute name="type" type="xs:string"/>
- <xs:attribute name="SuffixbaseForm" type="xs:string"/>
- <xs:attribute name="position" type="xs:string"/>
- <xs:attribute name="inflection" type="xs:string"/>
- </xs:complexType>
- </xs:element>
- </xs:sequence>
- <!-- stem-Attribute -->
- <xs:attribute name="type" type="xs:string"/>
- <xs:attribute name="pos" type="xs:string"/>
- <xs:attribute name="occurrence" type="xs:string"/>
- </xs:complexType>
- </xs:element>
- </xs:sequence>
- <!-- w -Attribute auf Wortebene -->
- <xs:attribute name="lemma" type="xs:string"/>
- <xs:attribute name="complexType" type="xs:string"/>
- <xs:attribute name="wordtype" type="xs:string"/>
- <xs:attribute name="occurrence" type="xs:string"/>
- <xs:attribute name="corpus" type="xs:string"/>
- <xs:attribute name="begin" type="xs:string"/>
- <xs:attribute name="end" type="xs:string"/>
- </xs:complexType>
- </xs:element>
- </xs:sequence>
- </xs:complexType>
- </xs:element>
- </xs:sequence>
- </element>
- <element name="wordtype" type="classification" minOccurs="0" maxOccurs="1">
- <classification id="wordtype"/>
- </element>
- <element name="complexType" type="classification" minOccurs="0" maxOccurs="1">
- <classification id="complexType"/>
- </element>
- <element name="corpus" type="classification" minOccurs="0" maxOccurs="1">
- <classification id="corpus"/>
- </element>
- <element name="pos" type="classification" minOccurs="0" maxOccurs="1">
- <classification id="pos"/>
- </element>
- <element name="PrefixbaseForm" type="classification" minOccurs="0"
- maxOccurs="1">
- <classification id="PrefixbaseForm"/>
- </element>
- <element name="SuffixbaseForm" type="classification" minOccurs="0"
- maxOccurs="1">
- <classification id="SuffixbaseForm"/>
- </element>
- <element name="inflection" type="classification" minOccurs="0" maxOccurs="1">
- <classification id="inflection"/>
- </element>
- <element name="corpuslink" type="link" minOccurs="0" maxOccurs="unbounded" >
- <target type="corpmeta"/>
- </element>
- </metadata>
-</objecttype>
-\end{lstlisting}
+*w* (line 17).
+
+.. _worddatamodel:
+
+.. code-block:: xml
+ :caption: Word Data Model
+ :linenos:
+ :emphasize-lines: 17
+
+ <?xml version="1.0" encoding="UTF-8"?>
+ <objecttype
+ name="morphilo"
+ isChild="true"
+ isParent="true"
+ hasDerivates="true"
+ xmlns:xs="http://www.w3.org/2001/XMLSchema"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:noNamespaceSchemaLocation="datamodel.xsd">
+ <metadata>
+ <element name="morphiloContainer" type="xml" style="dontknow"
+ notinherit="true" heritable="false">
+ <xs:sequence>
+ <xs:element name="morphilo">
+ <xs:complexType>
+ <xs:sequence>
+ <xs:element name="w" minOccurs="0" maxOccurs="unbounded">
+ <xs:complexType mixed="true">
+ <xs:sequence>
+ <!-- stem -->
+ <xs:element name="m1" minOccurs="0" maxOccurs="unbounded">
+ <xs:complexType mixed="true">
+ <xs:sequence>
+ <!-- base -->
+ <xs:element name="m2" minOccurs="0" maxOccurs="unbounded">
+ <xs:complexType mixed="true">
+ <xs:sequence>
+ <!-- root -->
+ <xs:element name="m3" minOccurs="0" maxOccurs="unbounded">
+ <xs:complexType mixed="true">
+ <xs:attribute name="type" type="xs:string"/>
+ </xs:complexType>
+ </xs:element>
+ <!-- prefix -->
+ <xs:element name="m4" minOccurs="0" maxOccurs="unbounded">
+ <xs:complexType mixed="true">
+ <xs:attribute name="type" type="xs:string"/>
+ <xs:attribute name="PrefixbaseForm" type="xs:string"/>
+ <xs:attribute name="position" type="xs:string"/>
+ </xs:complexType>
+ </xs:element>
+ </xs:sequence>
+ <xs:attribute name="type" type="xs:string"/>
+ </xs:complexType>
+ </xs:element>
+ <!-- suffix -->
+ <xs:element name="m5" minOccurs="0" maxOccurs="unbounded">
+ <xs:complexType mixed="true">
+ <xs:attribute name="type" type="xs:string"/>
+ <xs:attribute name="SuffixbaseForm" type="xs:string"/>
+ <xs:attribute name="position" type="xs:string"/>
+ <xs:attribute name="inflection" type="xs:string"/>
+ </xs:complexType>
+ </xs:element>
+ </xs:sequence>
+ <!-- stem-Attribute -->
+ <xs:attribute name="type" type="xs:string"/>
+ <xs:attribute name="pos" type="xs:string"/>
+ <xs:attribute name="occurrence" type="xs:string"/>
+ </xs:complexType>
+ </xs:element>
+ </xs:sequence>
+ <!-- w -Attribute auf Wortebene -->
+ <xs:attribute name="lemma" type="xs:string"/>
+ <xs:attribute name="complexType" type="xs:string"/>
+ <xs:attribute name="wordtype" type="xs:string"/>
+ <xs:attribute name="occurrence" type="xs:string"/>
+ <xs:attribute name="corpus" type="xs:string"/>
+ <xs:attribute name="begin" type="xs:string"/>
+ <xs:attribute name="end" type="xs:string"/>
+ </xs:complexType>
+ </xs:element>
+ </xs:sequence>
+ </xs:complexType>
+ </xs:element>
+ </xs:sequence>
+ </element>
+ <element name="wordtype" type="classification" minOccurs="0" maxOccurs="1">
+ <classification id="wordtype"/>
+ </element>
+ <element name="complexType" type="classification" minOccurs="0" maxOccurs="1">
+ <classification id="complexType"/>
+ </element>
+ <element name="corpus" type="classification" minOccurs="0" maxOccurs="1">
+ <classification id="corpus"/>
+ </element>
+ <element name="pos" type="classification" minOccurs="0" maxOccurs="1">
+ <classification id="pos"/>
+ </element>
+ <element name="PrefixbaseForm" type="classification" minOccurs="0"
+ maxOccurs="1">
+ <classification id="PrefixbaseForm"/>
+ </element>
+ <element name="SuffixbaseForm" type="classification" minOccurs="0"
+ maxOccurs="1">
+ <classification id="SuffixbaseForm"/>
+ </element>
+ <element name="inflection" type="classification" minOccurs="0" maxOccurs="1">
+ <classification id="inflection"/>
+ </element>
+ <element name="corpuslink" type="link" minOccurs="0" maxOccurs="unbounded" >
+ <target type="corpmeta"/>
+ </element>
+ </metadata>
+ </objecttype>
Additionally, it is worth mentioning that some attributes are modeled as a
-\emph{classification}. All these have to be listed
+*classification*. All these have to be listed
as separate elements in the data model. This has been done for all attributes
that are more or less subject to little or no change. In fact, all known suffix
and prefix morphemes should be known for the language investigated and are
therefore defined as a classification.
-The same is true for the parts of speech named \emph{pos} in the morphilo data
+The same is true for the parts of speech named *pos* in the morphilo data
model above.
Here the PENN-Treebank tagset was used. Last, the different morphemic layers in
-the standard model named \emph{m} are changed to $m1$ through $m5$. This is the
+the standard model named *m* are changed to *m1* through *m5*. This is the
only change in the standard that could be problematic if the data is to be
processed elsewhere and the change is not documented more explicitly. Yet, this
change was necessary for the MyCoRe repository throws errors caused by ambiguity
-issues on the different $m$-layers.
+issues on the different *m*-layers.
The second data model describes only very few properties of the text corpora
-from which the words are extracted. Listing \ref{lst:corpusdatamodel} depicts
+from which the words are extracted. Listing :ref:`corpusdatamodel` depicts
only the meta data element. For the sake of simplicity of the prototype, this
data model is kept as simple as possible. The obligatory field is the name of
the corpus. Specific dates of the corpus are classified as optional because in
some cases a text cannot be dated reliably.
-
-\begin{lstlisting}[language=XML,caption={Corpus Data
-Model},label=lst:corpusdatamodel]
-<metadata>
- <!-- Pflichtfelder -->
- <element name="korpusname" type="text" minOccurs="1" maxOccurs="1"/>
- <!-- Optionale Felder -->
- <element name="sprache" type="text" minOccurs="0" maxOccurs="1"/>
- <element name="size" type="number" minOccurs="0" maxOccurs="1"/>
- <element name="datefrom" type="text" minOccurs="0" maxOccurs="1"/>
- <element name="dateuntil" type="text" minOccurs="0" maxOccurs="1"/>
- <!-- number of words -->
- <element name="NoW" type="text" minOccurs="0" maxOccurs="1"/>
- <element name="corpuslink" type="link" minOccurs="0" maxOccurs="unbounded">
- <target type="morphilo"/>
- </element>
-</metadata>
-\end{lstlisting}
+.. _corpusdatamodel:
+
+.. code-block:: xml
+ :caption: Corpus Data Model
+
+ <metadata>
+ <!-- Pflichtfelder -->
+ <element name="korpusname" type="text" minOccurs="1" maxOccurs="1"/>
+ <!-- Optionale Felder -->
+ <element name="sprache" type="text" minOccurs="0" maxOccurs="1"/>
+ <element name="size" type="number" minOccurs="0" maxOccurs="1"/>
+ <element name="datefrom" type="text" minOccurs="0" maxOccurs="1"/>
+ <element name="dateuntil" type="text" minOccurs="0" maxOccurs="1"/>
+ <!-- number of words -->
+ <element name="NoW" type="text" minOccurs="0" maxOccurs="1"/>
+ <element name="corpuslink" type="link" minOccurs="0" maxOccurs="unbounded">
+ <target type="morphilo"/>
+ </element>
+ </metadata>
As a final remark, one might have noticed that all attributes are modelled as
strings although other data types are available and fields encoding the dates or
the number of words suggest otherwise. The MyCoRe framework even provides a
-data type \emph{historydate}. There is not a very satisfying answer to its
+data type *historydate*. There is not a very satisfying answer to its
disuse.
All that can be said is that the use of data types different than the string
leads later on to problems in the convergence between the search engine and the
repository framework. These issues seem to be well known and can be followed on
-github.
\ No newline at end of file
+`github <https://github.com/MyCoRe-Org>`_.
\ No newline at end of file
diff --git a/Morphilo_doc/_build/html/_sources/source/framework.rst.txt b/Morphilo_doc/_build/html/_sources/source/framework.rst.txt
index 1b9925de0b23715d8a34406bc9994664224637f3..aaac0021181c314bcddf879ed66834f1aaa5a9c2 100644
--- a/Morphilo_doc/_build/html/_sources/source/framework.rst.txt
+++ b/Morphilo_doc/_build/html/_sources/source/framework.rst.txt
@@ -1,27 +1,30 @@
Framework
=========
-\begin{figure}
- \centering
- \includegraphics[scale=0.33]{mycore_architecture-2.png}
- \caption[MyCoRe-Architecture and Components]{MyCoRe-Architecture and Components\protect\footnotemark}
- \label{fig:abbMyCoReStruktur}
-\end{figure}
-\footnotetext{source: https://www.mycore.de}
+.. figure:: images/mycore_architecture-2.png
+
+ Figure 2: MyCoRe-Architecture and Components [#f1]_
+
To specify the MyCoRe framework the morphilo application logic will have to be implemented,
-the TEI data model specified, and the input, search and output mask programmed.
+a data model specified, and the input, search and output mask programmed.
There are three directories which are
-important for adjusting the MyCoRe framework to the needs of one's own application. These three directories
+important for adjusting the MyCoRe framework to the needs of one's own application.
+
+These three directories
correspond essentially to the three components in the MVC model as explicated in
section \ref{subsec:mvc}. Roughly, they are envisualized in figure \ref{fig:abbMyCoReStruktur} in the upper
right hand corner. More precisely, the view (\emph{Layout} in figure \ref{fig:abbMyCoReStruktur}) and the model layer
(\emph{Datenmodell} in figure \ref{fig:abbMyCoReStruktur}) can be done
-completely via the ``interface'', which is a directory with a predefined
+completely via the *interface*, which is a directory with a predefined
structure and some standard files. For the configuration of the logic an extra directory is offered (/src/main/java/custom/mycore/addons/). Here all, java classes
extending the controller layer should be added.
Practically, all three MVC layers are placed in the
\emph{src/main/}-directory of the application. In one of the subdirectories,
\emph{datamodel/def}, the datamodel specifications are defined as xml files. It parallels the model
layer in the MVC pattern. How the data model was defined will be explained in
-section \ref{subsec:datamodelimpl}.
\ No newline at end of file
+section \ref{subsec:datamodelimpl}.
+
+.. rubric:: Notes
+
+.. [#f1] source: https://www.mycore.de
\ No newline at end of file
diff --git a/Morphilo_doc/_build/html/_sources/source/view.rst.txt b/Morphilo_doc/_build/html/_sources/source/view.rst.txt
index 5f09e06bd9d7a0d9c1edd889d8ac44d3cb36757f..5be49b3bb1ed98e41cd77c1b94439cce49e50051 100644
--- a/Morphilo_doc/_build/html/_sources/source/view.rst.txt
+++ b/Morphilo_doc/_build/html/_sources/source/view.rst.txt
@@ -4,244 +4,254 @@ View
Conceptualization
-----------------
-Lastly, the third directory (\emph{src/main/resources}) contains all code needed
+The MyCoRe-directory (*src/main/resources*) contains all code needed
for rendering the data to be displayed on the screen. So this corresponds to
the view in an MVC approach. It is done by xsl-files that (unfortunately)
contain some logic that really belongs to the controller. Thus, the division is
-not as clear as implied in theory. I will discuss this issue more specifically in the
-relevant subsection below. Among the resources are also all images, styles, and
+not as clear as implied in theory. I will point at this issue more specifically in the
+relevant subsection below. Among the resources are all images, styles, and
javascripts.
Implementation
--------------
-As explained in section \ref{subsec:mvc}, the view component handles the visual
+The view component handles the visual
representation in the form of an interface that allows interaction between
the user and the task to be carried out by the machine. As a
webservice in the present case, all interaction happens via a browser, i.e. webpages are
visualized and responses are recognized by registering mouse or keyboard
events. More specifically, a webpage is rendered by transforming xml documents
to html pages. The MyCoRe repository framework uses an open source XSLT
-processor from Apache, Xalan.\footnote{http://xalan.apache.org} This engine
+processor from Apache, `Xalan <http://xalan.apache.org>`_. This engine
transforms document nodes described by the XPath syntax into hypertext making
use of a special form of template matching. All templates are collected in so
called xml-encoded stylesheets. Since there are two data models with two
different structures, it is good practice to define two stylesheet files one for
each data model.
-As a demonstration, in listing \ref{lst:morphilostylesheet} below a short
+As a demonstration, in the listing below a short
extract is given for rendering the word data.
-\begin{lstlisting}[language=XML,caption={stylesheet
-morphilo.xsl},label=lst:morphilostylesheet]
-<?xml version="1.0" encoding="UTF-8"?>
-<xsl:stylesheet
- xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
- xmlns:xalan="http://xml.apache.org/xalan"
- xmlns:i18n="xalan://org.mycore.services.i18n.MCRTranslation"
- xmlns:acl="xalan://org.mycore.access.MCRAccessManager"
- xmlns:mcr="http://www.mycore.org/" xmlns:xlink="http://www.w3.org/1999/xlink"
- xmlns:mods="http://www.loc.gov/mods/v3"
- xmlns:encoder="xalan://java.net.URLEncoder"
- xmlns:mcrxsl="xalan://org.mycore.common.xml.MCRXMLFunctions"
- xmlns:mcrurn="xalan://org.mycore.urn.MCRXMLFunctions"
- exclude-result-prefixes="xalan xlink mcr i18n acl mods mcrxsl mcrurn encoder"
- version="1.0">
- <xsl:param name="MCR.Users.Superuser.UserName"/>
-
- <xsl:template match="/mycoreobject[contains(@ID,'_morphilo_')]">
- <head>
- <link href="{$WebApplicationBaseURL}css/file.css" rel="stylesheet"/>
- </head>
- <div class="row">
- <xsl:call-template name="objectAction">
- <xsl:with-param name="id" select="@ID"/>
- <xsl:with-param name="deriv" select="structure/derobjects/derobject/@xlink:href"/>
- </xsl:call-template>
- <xsl:variable name="objID" select="@ID"/>
- <!-- Hier Ueberschrift setzen -->
- <h1 style="text-indent: 4em;">
- <xsl:if test="metadata/def.morphiloContainer/morphiloContainer/morphilo/w">
- <xsl:value-of select="metadata/def.morphiloContainer/morphiloContainer/morphilo/w/text()[string-length(normalize-space(.))>0]"/>
- </xsl:if>
- </h1>
- <dl class="dl-horizontal">
- <!-- (1) Display word -->
- <xsl:if test="metadata/def.morphiloContainer/morphiloContainer/morphilo/w">
- <dt>
- <xsl:value-of select="i18n:translate('response.page.label.word')"/>
- </dt>
- <dd>
- <xsl:value-of select="metadata/def.morphiloContainer/morphiloContainer/morphilo/w/text()[string-length(normalize-space(.))>0]"/>
- </dd>
- </xsl:if>
- <!-- (2) Display lemma -->
- ...
- </xsl:template>
- ...
- <xsl:template name="objectAction">
- ...
- </xsl:template>
-...
-</xsl:stylesheet>
-\end{lstlisting}
+.. code-block:: xml
+ :caption: word data rendering in morphilo.xsl
+ :name: morphilo.xsl
+
+ <?xml version="1.0" encoding="UTF-8"?>
+ <xsl:stylesheet
+ xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
+ xmlns:xalan="http://xml.apache.org/xalan"
+ xmlns:i18n="xalan://org.mycore.services.i18n.MCRTranslation"
+ xmlns:acl="xalan://org.mycore.access.MCRAccessManager"
+ xmlns:mcr="http://www.mycore.org/" xmlns:xlink="http://www.w3.org/1999/xlink"
+ xmlns:mods="http://www.loc.gov/mods/v3"
+ xmlns:encoder="xalan://java.net.URLEncoder"
+ xmlns:mcrxsl="xalan://org.mycore.common.xml.MCRXMLFunctions"
+ xmlns:mcrurn="xalan://org.mycore.urn.MCRXMLFunctions" exclude-result-prefixes="xalan xlink mcr i18n acl mods mcrxsl mcrurn encoder" version="1.0">
+ <xsl:param name="MCR.Users.Superuser.UserName"/>
+ <xsl:template match="/mycoreobject[contains(@ID,'_morphilo_')]">
+ <head>
+ <link href="{$WebApplicationBaseURL}css/file.css" rel="stylesheet"/>
+ </head>
+ <div class="row">
+ <xsl:call-template name="objectAction">
+ <xsl:with-param name="id" select="@ID"/>
+ <xsl:with-param name="deriv" select="structure/derobjects/derobject/@xlink:href"/>
+ </xsl:call-template>
+ <xsl:variable name="objID" select="@ID"/>
+ <!-- Hier Ueberschrift setzen -->
+ <h1 style="text-indent: 4em;">
+ <xsl:if test="metadata/def.morphiloContainer/morphiloContainer/morphilo/w">
+ <xsl:value-of select="metadata/def.morphiloContainer/morphiloContainer/morphilo/w/text()[string-length(normalize-space(.))>0]"/>
+ </xsl:if>
+ </h1>
+ <dl class="dl-horizontal">
+ <!-- (1) Display word -->
+ <xsl:if test="metadata/def.morphiloContainer/morphiloContainer/morphilo/w">
+ <dt>
+ <xsl:value-of select="i18n:translate('response.page.label.word')"/>
+ </dt>
+ <dd>
+ <xsl:value-of select="metadata/def.morphiloContainer/morphiloContainer/morphilo/w/text()[string-length(normalize-space(.))>0]"/>
+ </dd>
+ </xsl:if>
+ <!-- (2) Display lemma -->
+ ...
+ </xsl:template>
+ ...
+ <xsl:template name="objectAction">
+ ...
+ </xsl:template>
+ ...
+ </xsl:stylesheet>
+
+
This template matches with
-the root node of each \emph{MyCoRe object} ensuring that a valid MyCoRe model is
+the root node of each *MyCoRe object* ensuring that a valid MyCoRe model is
used and checking that the document to be processed contains a unique
-identifier, here a \emph{MyCoRe-ID}, and the name of the correct data model,
-here \emph{morphilo}.
-Then, another template, \emph{objectAction}, is called together with two parameters, the ids
+identifier, here a *MyCoRe-ID*, and the name of the correct data model,
+here *morphilo*.
+Then, another template, *objectAction*, is called together with two parameters, the ids
of the document object and attached files. In the remainder all relevant
information from the document is accessed by XPath, such as the word and the lemma,
and enriched with hypertext annotations it is rendered as a hypertext document.
-The template \emph{objectAction} is key to understand the coupling process in the software
-framework. It is therefore separately listed in \ref{lst:objActionTempl}.
-
-\begin{lstlisting}[language=XML,caption={template
-objectAction},label=lst:objActionTempl,escapechar=|]
-<xsl:template name="objectAction">
- <xsl:param name="id" select="./@ID"/>
- <xsl:param name="accessedit" select="acl:checkPermission($id,'writedb')"/>
- <xsl:param name="accessdelete" select="acl:checkPermission($id,'deletedb')"/>
- <xsl:variable name="derivCorp" select="./@label"/>
- <xsl:variable name="corpID" select="metadata/def.corpuslink[@class='MCRMetaLinkID']/corpuslink/@xlink:href"/>
- <xsl:if test="$accessedit or $accessdelete">|\label{ln:ng}|
- <div class="dropdown pull-right">
- <xsl:if test="string-length($corpID) > 0 or $CurrentUser='administrator'">
- <button class="btn btn-default dropdown-toggle" style="margin:10px" type="button" id="dropdownMenu1" data-toggle="dropdown" aria-expanded="true">
- <span class="glyphicon glyphicon-cog" aria-hidden="true"></span> Annotieren
- <span class="caret"></span>
- </button>
- </xsl:if>
- <xsl:if test="string-length($corpID) > 0">|\label{ln:ru}|
- <xsl:variable name="ifsDirectory" select="document(concat('ifs:/',$derivCorp))"/>
- <ul class="dropdown-menu" role="menu" aria-labelledby="dropdownMenu1">
- <li role="presentation">
- |\label{ln:nw1}|<a href="{$ServletsBaseURL}object/tag{$HttpSession}?id={$derivCorp}&objID={$corpID}" role="menuitem" tabindex="-1">|\label{ln:nw2}|
- <xsl:value-of select="i18n:translate('object.nextObject')"/>
- </a>
- </li>
- <li role="presentation">
- <a href="{$WebApplicationBaseURL}receive/{$corpID}" role="menuitem" tabindex="-1">
- <xsl:value-of select="i18n:translate('object.backToProject')"/>
- </a>
- </li>
- </ul>
- </xsl:if>
- <xsl:if test="$CurrentUser='administrator'">
- <ul class="dropdown-menu" role="menu" aria-labelledby="dropdownMenu1">
- <li role="presentation">
- <a role="menuitem" tabindex="-1" href="{$WebApplicationBaseURL}content/publish/morphilo.xed?id={$id}">
- <xsl:value-of select="i18n:translate('object.editWord')"/>
- </a>
- </li>
- <li role="presentation">
- <a href="{$ServletsBaseURL}object/delete{$HttpSession}?id={$id}" role="menuitem" tabindex="-1" class="confirm_deletion option" data-text="Wirklich loeschen">
- <xsl:value-of select="i18n:translate('object.delWord')"/>
- </a>
- </li>
- </ul>
- </xsl:if>
- </div>
- <div class="row" style="margin-left:0px; margin-right:10px">
- <xsl:apply-templates select="structure/derobjects/derobject[acl:checkPermission(@xlink:href,'read')]">
- <xsl:with-param name="objID" select="@ID"/>
- </xsl:apply-templates>
- </div>
- </xsl:if>
-</xsl:template>
-\end{lstlisting}
-The \emph{objectAction} template defines the selection menu appearing -- once manual tagging has
+The template *objectAction* is key to understand the coupling process in the software
+framework. It is therefore separately listed in :ref:`objActionTempl`.
+
+.. _objActionTempl:
+
+.. code-block:: xml
+ :caption: template ObjectAction
+ :linenos:
+ :emphasize-lines: 7, 15, 19
+
+ <xsl:template name="objectAction">
+ <xsl:param name="id" select="./@ID"/>
+ <xsl:param name="accessedit" select="acl:checkPermission($id,'writedb')"/>
+ <xsl:param name="accessdelete" select="acl:checkPermission($id,'deletedb')"/>
+ <xsl:variable name="derivCorp" select="./@label"/>
+ <xsl:variable name="corpID" select="metadata/def.corpuslink[@class='MCRMetaLinkID']/corpuslink/@xlink:href"/>
+ <xsl:if test="$accessedit or $accessdelete">
+ <div class="dropdown pull-right">
+ <xsl:if test="string-length($corpID) > 0 or $CurrentUser='administrator'">
+ <button class="btn btn-default dropdown-toggle" style="margin:10px" type="button" id="dropdownMenu1" data-toggle="dropdown" aria-expanded="true">
+ <span class="glyphicon glyphicon-cog" aria-hidden="true"></span> Annotieren
+ <span class="caret"></span>
+ </button>
+ </xsl:if>
+ <xsl:if test="string-length($corpID) > 0">
+ <xsl:variable name="ifsDirectory" select="document(concat('ifs:/',$derivCorp))"/>
+ <ul class="dropdown-menu" role="menu" aria-labelledby="dropdownMenu1">
+ <li role="presentation">
+ <a href="{$ServletsBaseURL}object/tag{$HttpSession}?id={$derivCorp}&objID={$corpID}" role="menuitem" tabindex="-1">
+ <xsl:value-of select="i18n:translate('object.nextObject')"/>
+ </a>
+ </li>
+ <li role="presentation">
+ <a href="{$WebApplicationBaseURL}receive/{$corpID}" role="menuitem" tabindex="-1">
+ <xsl:value-of select="i18n:translate('object.backToProject')"/>
+ </a>
+ </li>
+ </ul>
+ </xsl:if>
+ <xsl:if test="$CurrentUser='administrator'">
+ <ul class="dropdown-menu" role="menu" aria-labelledby="dropdownMenu1">
+ <li role="presentation">
+ <a role="menuitem" tabindex="-1" href="{$WebApplicationBaseURL}content/publish/morphilo.xed?id={$id}">
+ <xsl:value-of select="i18n:translate('object.editWord')"/>
+ </a>
+ </li>
+ <li role="presentation">
+ <a href="{$ServletsBaseURL}object/delete{$HttpSession}?id={$id}" role="menuitem" tabindex="-1" class="confirm_deletion option" data-text="Wirklich loeschen">
+ <xsl:value-of select="i18n:translate('object.delWord')"/>
+ </a>
+ </li>
+ </ul>
+ </xsl:if>
+ </div>
+ <div class="row" style="margin-left:0px; margin-right:10px">
+ <xsl:apply-templates select="structure/derobjects/derobject[acl:checkPermission(@xlink:href,'read')]">
+ <xsl:with-param name="objID" select="@ID"/>
+ </xsl:apply-templates>
+ </div>
+ </xsl:if>
+ </xsl:template>
+
+The *objectAction* template defines the selection menu appearing -- once manual tagging has
started -- on the upper right hand side of the webpage entitled
-\emph{Annotieren} and displaying the two options \emph{next word} or \emph{back
-to project}.
-The first thing to note here is that in line \ref{ln:ng} a simple test
+*Annotieren* and displaying the two options *next word* or *back
+to project*.
+The first thing to note here is that in line 7 a simple test
excludes all guest users from accessing the procedure. After ensuring that only
-the user who owns the corpus project has access (line \ref{ln:ru}), s/he will be
+the user who owns the corpus project has access (line 15), s/he will be
able to access the drop down menu, which is really a url, e.g. line
-\ref{ln:nw1}. The attentive reader might have noticed that
-the url exactly matches the definition in the web-fragment.xml as shown in
-listing \ref{lst:webfragment}, line \ref{ln:tag}, which resolves to the
+19. The attentive reader might have noticed that
+the url exactly matches the definition in the *web-fragment.xml* as shown in
+listing :ref:`webxml`, line 17, which resolves to the
respective java class there. Really, this mechanism is the data interface within the
-MVC pattern. The url also contains two variables, named \emph{derivCorp} and
-\emph{corpID}, that are needed to identify the corpus and file object by the
-java classes (see section \ref{sec:javacode}).
+MVC pattern. The url also contains two variables, named *derivCorp* and
+*corpID*, that are needed to identify the corpus and file object by the
+java classes (see section :ref:`controller-section`).
The morphilo.xsl stylesheet contains yet another modification that deserves mention.
-In listing \ref{lst:derobjectTempl}, line \ref{ln:morphMenu}, two menu options --
-\emph{Tag automatically} and \emph{Tag manually} -- are defined. The former option
-initiates ProcessCorpusServlet.java as can be seen again in listing \ref{lst:webfragment},
-line \ref{ln:process}, which determines words that are not in the master data base.
+In listing :ref:`derobjectTempl`, line 18, two menu options --
+*Tag automatically* and *Tag manually* -- are defined. The former option
+initiates ProcessCorpusServlet.java as can be seen again in listing :ref:`webxml`,
+line 7, which determines words that are not in the master data base.
Still, it is important to note that the menu option is only displayed if two restrictions
-are met. First, a file has to be uploaded (line \ref{ln:1test}) and, second, there must be
+are met. First, a file has to be uploaded (line 19) and, second, there must be
only one file. This is necessary because in the annotation process other files will be generated
that store the words that were not yet processed or a file that includes the final result. The
generated files follow a certain pattern. The file harboring the final, entire TEI-annotated
-corpus is prefixed by \emph{tagged}, the other file is prefixed \emph{untagged}. This circumstance
-is exploited for manipulating the second option (line \ref{ln:loop}). A loop runs through all
-files in the respective directory and if a file name starts with \emph{untagged},
+corpus is prefixed by *tagged*, the other file is prefixed *untagged*. This circumstance
+is exploited for manipulating the second option (line 27). A loop runs through all
+files in the respective directory and if a file name starts with *untagged*,
the option to manually tag is displayed.
-\begin{lstlisting}[language=XML,caption={template
-matching derobject},label=lst:derobjectTempl,escapechar=|]
-<xsl:template match="derobject" mode="derivateActions">
- <xsl:param name="deriv" />
- <xsl:param name="parentObjID" />
- <xsl:param name="suffix" select="''" />
- <xsl:param name="id" select="../../../@ID" />
- <xsl:if test="acl:checkPermission($deriv,'writedb')">
- <xsl:variable name="ifsDirectory" select="document(concat('ifs:',$deriv,'/'))" />
- <xsl:variable name="path" select="$ifsDirectory/mcr_directory/path" />
- ...
- <div class="options pull-right">
- <div class="btn-group" style="margin:10px">
- <a href="#" class="btn btn-default dropdown-toggle" data-toggle="dropdown">
- <i class="fa fa-cog"></i>
- <xsl:value-of select="' Korpus'"/>
- <span class="caret"></span>
- </a>
- <ul class="dropdown-menu dropdown-menu-right">
- <!-- Anpasssungen Morphilo -->|\label{ln:morphMenu}|
- <xsl:if test="string-length($deriv) > 0">|\label{ln:1test}|
- <xsl:if test="count($ifsDirectory/mcr_directory/children/child) = 1">|\label{ln:2test}|
- <li role="presentation">
- <a href="{$ServletsBaseURL}object/process{$HttpSession}?id={$deriv}&objID={$id}" role="menuitem" tabindex="-1">
- <xsl:value-of select="i18n:translate('derivate.process')"/>
- </a>
- </li>
- </xsl:if>
- <xsl:for-each select="$ifsDirectory/mcr_directory/children/child">|\label{ln:loop}|
- <xsl:variable name="untagged" select="concat($path, 'untagged')"/>
- <xsl:variable name="filename" select="concat($path,./name)"/>
- <xsl:if test="starts-with($filename, $untagged)">
- <li role="presentation">
- <a href="{$ServletsBaseURL}object/tag{$HttpSession}?id={$deriv}&objID={$id}" role="menuitem" tabindex="-1">
- <xsl:value-of select="i18n:translate('derivate.taggen')"/>
- </a>
- </li>
+.. _derobjectTempl:
+
+.. code-block:: xml
+ :caption: derobject template
+ :linenos:
+ :emphasize-lines: 18,19, 27
+
+ <xsl:template match="derobject" mode="derivateActions">
+ <xsl:param name="deriv" />
+ <xsl:param name="parentObjID" />
+ <xsl:param name="suffix" select="''" />
+ <xsl:param name="id" select="../../../@ID" />
+ <xsl:if test="acl:checkPermission($deriv,'writedb')">
+ <xsl:variable name="ifsDirectory" select="document(concat('ifs:',$deriv,'/'))" />
+ <xsl:variable name="path" select="$ifsDirectory/mcr_directory/path" />
+ ...
+ <div class="options pull-right">
+ <div class="btn-group" style="margin:10px">
+ <a href="#" class="btn btn-default dropdown-toggle" data-toggle="dropdown">
+ <i class="fa fa-cog"></i>
+ <xsl:value-of select="' Korpus'"/>
+ <span class="caret"></span>
+ </a>
+ <ul class="dropdown-menu dropdown-menu-right">
+ <!-- Anpasssungen Morphilo -->|\label{ln:morphMenu}|
+ <xsl:if test="string-length($deriv) > 0">|\label{ln:1test}|
+ <xsl:if test="count($ifsDirectory/mcr_directory/children/child) = 1">|\label{ln:2test}|
+ <li role="presentation">
+ <a href="{$ServletsBaseURL}object/process{$HttpSession}?id={$deriv}&objID={$id}" role="menuitem" tabindex="-1">
+ <xsl:value-of select="i18n:translate('derivate.process')"/>
+ </a>
+ </li>
+ </xsl:if>
+ <xsl:for-each select="$ifsDirectory/mcr_directory/children/child">|\label{ln:loop}|
+ <xsl:variable name="untagged" select="concat($path, 'untagged')"/>
+ <xsl:variable name="filename" select="concat($path,./name)"/>
+ <xsl:if test="starts-with($filename, $untagged)">
+ <li role="presentation">
+ <a href="{$ServletsBaseURL}object/tag{$HttpSession}?id={$deriv}&objID={$id}" role="menuitem" tabindex="-1">
+ <xsl:value-of select="i18n:translate('derivate.taggen')"/>
+ </a>
+ </li>
+ </xsl:if>
+ </xsl:for-each>
</xsl:if>
- </xsl:for-each>
- </xsl:if>
- ...
- </ul>
- </div>
- </div>
- </xsl:if>
-</xsl:template>
-\end{lstlisting}
-
-Besides the two stylesheets morphilo.xsl and corpmeta.xsl, other stylesheets have
+ ...
+ </ul>
+ </div>
+ </div>
+ </xsl:if>
+ </xsl:template>
+
+
+Besides the two stylesheets *morphilo.xsl* and *corpmeta.xsl*, other stylesheets have
to be adjusted. They will not be discussed in detail here for they are self-explanatory for the most part.
-Essentially, they render the overall layout (\emph{common-layout.xsl}, \emph{skeleton\_layout\_template.xsl})
+Essentially, they render the overall layout (*common-layout.xsl*, *skeleton_layout_template.xsl*)
or the presentation
-of the search results (\emph{response-page.xsl}) and definitions of the solr search fields (\emph{searchfields-solr.xsl}).
-The former and latter also inherit templates from \emph{response-general.xsl} and \emph{response-browse.xsl}, in which the
+of the search results (*response-page.xsl*) and definitions of the solr search fields (*searchfields-solr.xsl*).
+The former and latter also inherit templates from *response-general.xsl* and *response-browse.xsl*, in which the
navigation bar of search results can be changed. For the use of multilinguality a separate configuration directory
-has to be created containing as many \emph{.property}-files as different
-languages want to be displayed. In the current case these are restricted to German and English (\emph{messages\_de.properties} and \emph{messages\_en.properties}).
-The property files include all \emph{i18n} definitions. All these files are located in the \emph{resources} directory.
+has to be created containing as many *.property*-files as different
+languages want to be displayed. In the current case these are restricted to German and English (*messages_de.properties* and *messages_en.properties*).
+The property files include all *i18n* definitions. All these files are located in the *resources* directory.
Furthermore, a search mask and a page for manually entering the annotations had
to be designed.
-For these files a specially designed xml standard (\emph{xed}) is recommended to be used within the
+For these files a specially designed xml standard (*xed*) is recommended to be used within the
repository framework.
\ No newline at end of file
diff --git a/Morphilo_doc/_build/html/_static/architecture.pdf b/Morphilo_doc/_build/html/_static/architecture.pdf
deleted file mode 100644
index 2f671b5abd23245db733040b2e56873976a2f9fd..0000000000000000000000000000000000000000
Binary files a/Morphilo_doc/_build/html/_static/architecture.pdf and /dev/null differ
diff --git a/Morphilo_doc/_build/html/index.html b/Morphilo_doc/_build/html/index.html
index 91c11e71b709e6623eea1130842d039777641d21..289b24442a70a01df3a4f52c90eb4941aa1062c5 100644
--- a/Morphilo_doc/_build/html/index.html
+++ b/Morphilo_doc/_build/html/index.html
@@ -6,7 +6,7 @@
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>Documentation Morphilo Project — Morphilo documentation</title>
+ <title>Morphilo Project Documentation — Morphilo documentation</title>
<link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript" src="_static/documentation_options.js"></script>
@@ -15,7 +15,7 @@
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
- <link rel="next" title="Data Model" href="source/datamodel.html" />
+ <link rel="next" title="Software Design" href="source/architecture.html" />
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
@@ -30,32 +30,33 @@
<div class="bodywrapper">
<div class="body" role="main">
- <div class="section" id="documentation-morphilo-project">
-<h1>Documentation Morphilo Project<a class="headerlink" href="#documentation-morphilo-project" title="Permalink to this headline">¶</a></h1>
+ <div class="section" id="morphilo-project-documentation">
+<h1>Morphilo Project Documentation<a class="headerlink" href="#morphilo-project-documentation" title="Permalink to this headline">¶</a></h1>
<div class="toctree-wrapper compound">
<p class="caption"><span class="caption-text">Contents:</span></p>
<ul>
+<li class="toctree-l1"><a class="reference internal" href="source/architecture.html">Software Design</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="source/architecture.html#mvc-model">MVC Model</a></li>
+<li class="toctree-l2"><a class="reference internal" href="source/architecture.html#morphilo-architecture">Morphilo Architecture</a></li>
+<li class="toctree-l2"><a class="reference internal" href="source/architecture.html#repository-framework">Repository Framework</a></li>
+</ul>
+</li>
<li class="toctree-l1"><a class="reference internal" href="source/datamodel.html">Data Model</a><ul>
<li class="toctree-l2"><a class="reference internal" href="source/datamodel.html#conceptualization">Conceptualization</a></li>
<li class="toctree-l2"><a class="reference internal" href="source/datamodel.html#implementation">Implementation</a></li>
</ul>
</li>
-<li class="toctree-l1"><a class="reference internal" href="source/controller.html">Controller Adjustments</a><ul>
-<li class="toctree-l2"><a class="reference internal" href="source/controller.html#general-principle-of-operation">General Principle of Operation</a></li>
-<li class="toctree-l2"><a class="reference internal" href="source/controller.html#conceptualization">Conceptualization</a></li>
-<li class="toctree-l2"><a class="reference internal" href="source/controller.html#implementation">Implementation</a><ul>
-<li class="toctree-l3"><a class="reference internal" href="source/controller.html#id13">}</a></li>
-</ul>
-</li>
-</ul>
-</li>
<li class="toctree-l1"><a class="reference internal" href="source/view.html">View</a><ul>
<li class="toctree-l2"><a class="reference internal" href="source/view.html#conceptualization">Conceptualization</a></li>
<li class="toctree-l2"><a class="reference internal" href="source/view.html#implementation">Implementation</a></li>
</ul>
</li>
-<li class="toctree-l1"><a class="reference internal" href="source/architecture.html">Software Design</a></li>
-<li class="toctree-l1"><a class="reference internal" href="source/framework.html">Framework</a></li>
+<li class="toctree-l1"><a class="reference internal" href="source/controller.html">Controller Adjustments</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="source/controller.html#general-principle-of-operation">General Principle of Operation</a></li>
+<li class="toctree-l2"><a class="reference internal" href="source/controller.html#conceptualization">Conceptualization</a></li>
+<li class="toctree-l2"><a class="reference internal" href="source/controller.html#implementation">Implementation</a></li>
+</ul>
+</li>
</ul>
</div>
</div>
@@ -76,14 +77,14 @@
<div class="sphinxsidebarwrapper">
<h3><a href="#">Table Of Contents</a></h3>
<ul>
-<li><a class="reference internal" href="#">Documentation Morphilo Project</a></li>
+<li><a class="reference internal" href="#">Morphilo Project Documentation</a></li>
<li><a class="reference internal" href="#indices-and-tables">Indices and tables</a></li>
</ul>
<div class="relations">
<h3>Related Topics</h3>
<ul>
<li><a href="#">Documentation overview</a><ul>
- <li>Next: <a href="source/datamodel.html" title="next chapter">Data Model</a></li>
+ <li>Next: <a href="source/architecture.html" title="next chapter">Software Design</a></li>
</ul></li>
</ul>
</div>
diff --git a/Morphilo_doc/_build/html/objects.inv b/Morphilo_doc/_build/html/objects.inv
index ac6220ca7ddb74e3021a0040cda45c6da6630b09..7388eb80140c87ef9ab8f59417e02467513eff52 100644
Binary files a/Morphilo_doc/_build/html/objects.inv and b/Morphilo_doc/_build/html/objects.inv differ
diff --git a/Morphilo_doc/_build/html/searchindex.js b/Morphilo_doc/_build/html/searchindex.js
index c36d4c6073f0027ee03fec4a2a93fdb62ce9ab1d..340d50555f211bdc3e4ba05860f04af23e1d4003 100644
--- a/Morphilo_doc/_build/html/searchindex.js
+++ b/Morphilo_doc/_build/html/searchindex.js
@@ -1 +1 @@
-Search.setIndex({docnames:["index","source/architecture","source/controller","source/datamodel","source/framework","source/view"],envversion:52,filenames:["index.rst","source/architecture.rst","source/controller.rst","source/datamodel.rst","source/framework.rst","source/view.rst"],objects:{},objnames:{},objtypes:{},terms:{"0px":5,"10px":5,"1test":5,"2test":5,"4em":5,"abstract":3,"boolean":2,"case":[1,2,3,5],"catch":2,"class":[2,4,5],"default":[2,5],"enum":2,"final":[2,3,5],"function":[2,3],"import":[1,2,3,4,5],"int":2,"new":2,"null":2,"public":2,"return":2,"short":5,"throw":[2,3],"true":[1,2,3,5],"try":2,"void":2,"while":[1,2],AND:2,And:[1,2],For:[2,3,4,5],That:2,The:[1,2,3,5],Then:[2,5],There:[2,3,4],These:[2,3,4],Using:2,With:2,_morphilo_:5,abbmycorestruktur:4,abbruchbedingung:2,abl:[3,5],about:2,abov:[2,3],accept:2,access:[1,2,5],accessdelet:5,accessedit:5,accord:[2,3],accordingli:2,account:3,acl:5,actual:2,add:2,added:[2,3,4],adding:2,addit:[2,3],addition:3,addon:[2,4],address:2,addtomorphilodb:2,adject:3,adjust:[0,3,4,5],administ:1,administr:[2,5],admit:2,adopt:2,advantag:3,affix:[2,3],affixpo:2,affixstripp:2,affixtyp:2,after:[1,2,5],again:[2,5],agre:2,aim:2,al1:2,al2:2,algorithm:2,all:[1,2,3,4,5],allomorph:2,allow:[1,2,3,5],alon:2,alpha:1,alreadi:2,also:[1,2,3,5],altern:1,although:[2,3],alwai:2,ambigu:3,among:5,amp:5,analys:2,analysi:3,analyz:[2,3],analyzeinfl:2,analyzeinflect:2,analyzelemma:2,analyzepref:2,analyzeprefix:2,analyzeroot:2,analyzesuf:2,analyzesuffix:2,analyzeword:2,ani:[1,2],annot:[1,2,5],annotieren:5,anoth:[2,5],anpasssungen:5,answer:3,anzword:2,apach:[2,5],api:2,appear:[2,5],appli:[2,5],applic:[2,3,4],approach:[1,2,5],appropri:3,approxim:2,arbitrari:2,architect:1,architectur:[1,3,4],argument:2,aria:5,around:2,arrai:2,arraylist:2,asid:3,assert:2,assign:2,assum:[1,2],assumpt:[1,3],ation:2,attach:[3,5],attent:[3,5],attribut:[2,3],auf:3,aur:2,auslesen:2,authen:2,authentif:2,author:2,autom:1,automat:5,auver:2,avail:[1,2,3],avoid:2,awai:2,awkward:2,awver:2,back:[2,5],backtoproject:5,bar:5,base:[1,2,3,5],baseform:3,basi:2,basic:[2,3],becaus:[1,2,3,5],becom:[1,2],been:[2,3],befor:2,beforehand:1,begin:[2,3,4,5],beginnig:2,begintim:2,behind:2,being:2,belong:5,below:[2,5],besid:5,best:3,between:[1,2,3,5],beyond:2,bird:2,bit:2,both:[1,2,3],branch:[1,3],brows:5,browser:5,btn:5,bugfixend:2,bugfixstart:2,build:2,builder:2,button:5,calcul:[1,2],call:[1,2,5],callkeymeth:2,calll:2,can:[1,2,3,4,5],candid:2,cannot:[2,3],caption:[2,3,4,5],care:2,caret:5,carri:[1,2,3,5],caus:[2,3],center:[2,4],certain:5,chang:[2,3,5],changeabl:2,chap:2,chapter:2,check:[2,5],checkand:2,checkpermiss:5,child:5,children:5,choic:3,chosen:3,circumst:5,claim:2,classdiag:2,classid:2,classif:3,classifi:3,clear:[2,5],clearer:1,click:2,client:2,cmp:2,code:[2,5],cog:5,collect:[2,5],com:[2,3],come:2,comfort:3,common:5,compar:[1,2],comparemcr:2,comparemcrobject:2,compareto:2,comparison:[1,2],compat:3,compil:2,complet:[1,2,3,4],complex:2,complextyp:3,complic:2,compon:[1,2,4,5],componentsprotectfootnotemark:4,compound:2,compris:[2,3],con:3,concat:5,concaten:2,concept:[2,3],conceptu:0,concern:2,conclud:2,concret:1,condit:2,configur:[2,4,5],confirm:2,confirm_delet:5,congruenc:1,consequ:3,consid:2,consist:2,construct:[2,3],constructor:2,constructqc:2,contain:[1,2,5],content:[0,2,5],continu:1,control:[0,1,4,5],converg:3,core:2,corner:4,corp1derivid:2,corp2derivid:2,corpdoc:2,corpid:[2,5],corpmeta:[2,3,5],corpo1:2,corpo2:2,corpobj:2,corpora:[1,2,3],corpu:[1,2,3,5],corpusdatamodel:[2,3],corpusdiff:2,corpuslink:[2,3,5],corpuss:2,correct:[1,2,5],correl:2,correspond:[2,3,4,5],could:[1,2,3],count:[1,2,5],counter:2,coupl:5,cours:[1,2],creat:[1,2,5],createdbi:2,createmorphiloobject:2,createxml:2,creator:2,crpid1:2,crpid2:2,crucial:[1,2,3],css:5,current:5,currentus:[2,5],custom:[2,4],cut:2,data:[0,2,4,5],databas:[1,2],datamodel:[2,3,4],datamodelimpl:4,dataset:[1,2],date:[2,3],datefrom:[2,3],datefromcorp:2,datenmodel:4,dateuntil:[2,3],dateuntilcorp:2,deal:2,decid:1,decis:1,def:[2,4,5],defin:[1,2,3,4,5],definit:[2,5],degre:2,delet:[2,3,5],deleteaffix:2,deletedb:5,deliv:[1,2],delword:5,demonstr:[2,5],depend:2,depict:[2,3],depth:2,deriv:[1,2,3,5],derivateact:5,derivcorp:5,derivid:2,derobject:[2,5],derobjecttempl:5,describ:[1,2,3,5],deserv:5,design:[0,2,5],detail:[2,5],detaileddiff:2,determin:[1,2,5],develop:3,deviat:1,diachron:3,diagram:2,dictionari:2,diff:2,diffbegin:2,diffend:2,differ:[1,2,3,5],difflist:2,diffsum:2,dimens:3,direct:[2,3],directli:[1,3],directori:[2,4,5],discuss:[2,5],displai:[2,5],distribut:[1,2],disus:3,div:5,divis:5,doc:3,document:[1,2,3,5],doe:2,domin:3,done:[2,3,4,5],dontknow:3,doubl:2,down:[2,5],drop:5,dropdown:5,dropdownmenu1:5,due:2,each:[2,3,5],earlier:2,easi:3,easili:2,editword:5,effect:2,effici:[2,3],either:2,elem:2,element:[2,3],elig:3,elimin:2,elm:2,els:2,elsewher:3,eman:3,embed:2,embedaffix:2,embedded:2,emph:[1,2,3,4,5],empti:2,enabl:2,enclos:3,encod:[2,3,5],end:[2,3,4,5],endswith:2,endtim:2,engin:[2,3,5],english:[1,2,5],enough:1,enrich:5,ensur:5,ent:2,enter:5,entir:[2,5],entitl:5,entri:[1,2],entrynum:2,enumer:2,enumpref:2,envisu:4,equal:2,equalobject:2,equalocc2:2,equalocc:2,equival:2,err:2,error:[1,3],erzeugen:2,escapechar:[2,3,5],essenc:2,essenti:[4,5],establish:3,estim:1,etc:2,euer:2,european:3,evalu:2,evaluatefirst:2,even:3,event:5,everi:2,exact:[1,2],exactli:[1,2,5],exampl:[1,2,3],except:2,excerpt:2,exchang:[2,3],exclud:5,exist:2,expand:5,explain:[2,4,5],explan:2,explanatori:5,explic:4,explicit:2,explicitli:[2,3],exploit:5,exponenti:2,extend:[2,4],extent:3,extra:[1,2,4],extract:[2,3,5],extrem:2,eye:2,fact:[1,2,3],factori:2,fail:2,fall:2,fals:[2,3],felder:3,few:3,field:[2,3,5],fig:[1,2,4],figur:[1,2,4],file:[2,3,4,5],filenam:[2,5],filepathproc:2,filesaveend:2,filesavestart:2,fill:2,filter:2,find:2,first:[1,2,5],five:2,fix:2,flag:2,flow:1,focu:2,follow:[1,2,3,5],footnot:[2,3,5],footnotetext:4,form:[1,2,5],format:2,former:[2,5],fort:3,four:[2,3],fragment:[2,5],framework:[0,2,3,5],freeli:2,frequenc:[1,2],frequent:2,from:[1,2,3,5],fuer:2,fulfil:2,full:2,fulli:[1,2],further:[1,2],furthermor:[3,5],futur:3,gauss:1,gegeben:2,gener:[0,5],german:5,get:2,getaffixposit:2,getalldiffer:2,getallequalmcrobject:2,getattributevalu:2,getbaseurl:2,getcont:2,getcontentfromfil:2,getcontrolnodedetail:2,getcorp:2,getcorpusmetadata:2,getcurrentsess:2,getderiv:2,getderivatefilepath:2,getequalnumb:2,getequalobjectnumb:2,getfieldvalu:2,getfilenam:2,getid:2,getinst:2,getmaindocnam:2,getmorphem:2,getnamespac:2,getnumberofword:2,getnumfound:2,getoccurrencesfromequaltext:2,getpath:2,getprettyformat:2,getresult:2,getsolrcli:2,getstemnumb:2,getter:2,gettestnodedetail:2,gettext:2,getunknownword:2,getunknowword:2,geturl:2,geturlparamet:2,getuserid:2,getuserinform:2,getvalu:2,getxmlfromobject:2,getxpathloc:2,gist:1,github:3,give:2,given:[1,2,3,5],gleich:2,glyphicon:5,good:5,gov:5,greater:2,group:5,grow:[1,2],guest:5,guidelin:3,had:5,hand:[1,2,3,4,5],handl:[2,5],happen:[2,5],harbor:[2,5],has:[1,2,3,5],hasderiv:3,hash:2,hashmap:2,hauver:2,have:[1,2,3,4,5],head:5,heavili:2,help:1,helpobj:2,henc:2,her:2,here:[1,2,3,4,5],herit:3,hidden:[1,5],hier:5,hierarch:3,hierarchi:[2,3],higher:2,his:2,histor:3,historyd:3,hit:2,horizont:5,houyr:2,hover:2,how:[1,2,4],howev:[1,2,3],href:[2,5],html:[3,5],http:[2,3,4,5],httpsession:5,hundr:2,hypertext:5,i18n:5,idea:2,ideal:3,ident:2,identifi:[2,3,5],ids:[2,5],ifs:5,ifsdirectori:5,ignor:[2,3],illustr:[1,2],imag:5,impact:1,implement:[0,4],impli:[2,3,5],implicitli:2,importantli:2,improv:2,includ:[2,3,5],includegraph:[2,4],increas:2,increment:2,incrocc:2,indent:5,independ:1,index:[0,2],indic:2,indo:3,infl:2,inflect:[2,3],inflectionenum:2,inflenum:2,inform:[1,3,5],ingredi:2,inherit:5,initi:[2,5],inner:2,input:[1,4],inspir:2,instanc:[2,3],instanti:2,instead:1,instruct:2,integ:2,integr:2,intend:2,interact:[2,3,5],interest:[2,3],interfac:[2,4,5],interrupt:1,investig:[2,3],involv:2,ioexcept:2,ion:2,isauthor:2,ischild:3,isempti:2,isequ:2,ispar:3,issu:[3,5],ist:2,item:[1,2],itemlabel:2,iter:2,itm:2,its:[2,3],java:[2,4,5],javacod:5,javascript:5,jdm:2,jdom:2,jdomdoc:2,jdomdochelp:2,jdomobject:2,jdomorphilo:2,job:2,just:[1,2,3],keep:[2,3],kei:[2,5],kept:3,keyboard:5,keyset:2,kind:3,known:3,korpu:5,korpusnam:[2,3],label:[2,3,4,5],labelenumi:2,labelledbi:5,laid:3,landscap:2,languag:[1,2,3,5],larg:[1,2],larger:2,last:[2,3],lastli:5,later:[1,2,3],latter:[2,5],law:2,layer:[3,4],layout:[4,5],lead:3,least:2,left:[2,3,5],leftov:2,leftsov:2,lemma:[2,3,5],lemmaanalyz:2,length:[2,5],less:[2,3],let:2,letter:2,level:[1,2,3],lexic:[1,2,3],like:[1,2],limit:2,line:[2,3,5],linear:2,link:[1,2,3,5],list:[2,3,5],littl:3,loc:5,locat:5,loeschen:5,logic:[2,4,5],longer:2,look:[1,2],loop:[2,5],lower:2,lst:[2,3,5],lstlist:[2,3,5],machin:5,made:[1,2],main:[2,4,5],mainfil:2,mainli:2,major:3,make:[1,2,3,5],manag:[1,2,3],mani:[2,5],manipul:[2,5],manual:[1,2,5],map:2,margin:5,mark:[1,2],markup:3,mask:[2,4,5],master:[1,2,5],match:[1,2,5],materi:1,matter:2,maxoccur:[2,3],mcr:5,mcr_directori:5,mcraccessmanag:5,mcridstr:2,mcrmetadatamanag:2,mcrmetalinkid:5,mcrobj1:2,mcrobj2:2,mcrobj:2,mcrobject:2,mcrobjectid:2,mcrobjekt:2,mcrpath:2,mcrservlet:2,mcrservletjob:2,mcrsessionmgr:2,mcrsolrclientfactori:2,mcrtranslat:5,mcrurn:5,mcrxmlfunction:[2,5],mcrxsl:5,mdm:2,mean:[2,3],measur:2,mechan:[2,5],meet:3,ment:2,mention:[2,3,5],menu:[2,5],menuitem:5,mere:2,messages_d:5,messages_en:5,met:5,meta:[2,3],metadata:[1,2,3,5],method:2,middl:[1,2],might:[3,5],mileston:2,minoccur:3,mix:3,mod:5,mode:[2,5],model:[0,2,4,5],modif:5,modul:[0,3],modular:2,monomorphem:2,more:[2,3,4,5],moreov:2,morphem:[2,3],morphil:1,morphilo:[1,2,3,4,5],morphilo_uml:2,morphilocontain:[2,3,5],morphilostylesheet:5,morphmenu:5,morpholog:2,morphologicalsystem:3,most:[2,3,5],mous:5,multi:3,multilingu:5,multipl:2,must:[2,5],mvc:[3,4,5],mycor:[2,3,4,5],mycore_architectur:4,mycoreobject:[2,5],name:[2,3,5],namespac:2,natur:2,navig:5,necessari:[1,2,3,5],need:[2,3,4,5],neg:3,neither:2,net:5,newli:[1,2],newroot:2,next:[2,5],nextobject:5,nirgend:2,node:[2,3,5],nonamespaceschemaloc:3,nor:2,normal:[1,5],notavail:2,note:[1,2,5],notic:[3,5],notinherit:3,now:[2,3],number:[1,2,3],numberformatexcept:2,nur:2,nw1:5,nw2:5,oar:2,obaer:2,ober:2,obj:2,objactiontempl:5,object:[1,2,3,5],objectact:5,objectid:2,objecttyp:3,objid:[2,5],obligatori:3,obuh:2,obviou:2,occdiff:2,occur:[1,2],occurr:[2,3],oder:2,oed:2,oedfootnot:2,oer:2,ofaer:2,ofer:2,oferr:2,off:2,offer:[2,4],offerr:2,offr:2,ofir:2,ofor:2,ofowr:2,oger:2,oher:2,onc:[2,5],one:[2,3,4,5],ones:2,ongo:1,onli:[1,2,3,5],onward:2,ooer:2,oor:2,oouer:2,open:5,oper:0,oppos:2,optim:1,option:[2,3,5],optional:3,order:2,org:[2,3,5],origin:2,other:[1,2,3,5],otherwis:[1,2,3],ouer:2,ouir:2,our:2,out:[1,2,3,5],outer:2,output:[1,2,4],outputstr:2,outputt:2,ouuer:2,ouur:2,ouver:2,ouyr:2,ova:2,ovah:2,ovar:2,ove:2,over:2,overal:5,overrid:2,overwhelm:3,overwrit:2,overwritten:2,ovir:2,ovr:2,ovuh:2,ovur:2,ovver:2,ovyr:2,ower:2,owir:2,own:[4,5],owr:2,owuer:2,owur:2,owver:2,owwer:2,owyr:2,oxford:2,packag:2,page:[0,2,5],paid:3,par:3,parallel:4,param:5,paramet:[2,5],parentobjid:5,parseint:2,part:[2,3,5],partial:1,pass:2,path:[2,5],pattern:[2,3,4,5],pcs2:2,pcs:2,pdf:3,penn:3,persist:[1,3],perspect:2,pflichtfeld:3,phase:2,phonolog:3,place:[2,4],plain:1,plu:2,png:[2,4],point:[1,2],popul:1,pos:[2,3],posit:[2,3],possibl:[1,2,3],post:2,potenti:[2,3],practic:[4,5],precis:[2,4],predefin:4,predict:3,pref:2,prefcutoff:2,prefenum:2,prefer:2,prefix:[2,3,5],prefixallomorph:2,prefixbaseform:3,prefixenum:2,prefixmorphem:2,prefixnumb:2,prefloop:2,prefputmorph:2,present:[2,3,5],prevent:2,previou:[1,2],previous:[1,2],principl:0,printstacktrac:2,privat:2,probabl:2,problem:[2,3],problemat:3,proc:2,procedur:[2,5],process:[1,2,3,5],processcorpusservlet:[2,5],processor:5,procwd:2,product:[2,3],program:[2,4],progress:1,project:[2,5],prop:2,properti:[3,5],propos:2,protect:2,prototyp:[2,3],provid:[1,2,3],publish:[2,5],pull:5,put:2,putal:2,qry:2,qualiti:[1,2],qualitycontrol:2,quantif:3,queri:2,question:[2,3],rang:2,rather:2,reach:2,read:[2,3,5],reader:5,readi:2,realiz:3,realli:[2,5],reason:[1,2,3],receiv:[1,2,5],recht:2,recogn:[2,3,5],recognit:2,recommend:5,recurs:2,redirect:2,ref:[1,2,3,4,5],refactor:2,reflect:3,regard:3,region:1,regist:[2,5],reject:1,rekurs:2,rel:5,relat:2,releas:3,relev:[2,3,5],reliabl:3,remain:2,remaind:[2,5],remark:3,remov:2,removecont:2,render:[2,5],renewcommand:2,repeat:2,repetit:2,replac:2,repositori:[2,3,5],represent:[2,5],request:2,requir:3,research:[1,3],resolv:[2,5],resourc:[1,5],respect:[1,2,3,5],respond:2,respons:[2,5],rest:2,restart:2,restrict:5,restword:2,result:[1,2,3,5],resultset:2,resum:[1,2],retrievemcrobject:2,reveal:2,revers:2,right:[2,3,4,5],rise:2,role:[2,5],roll:2,root:[2,3,5],rootanalyz:2,roughli:4,row:5,rslt:2,rudimentari:2,run:[2,5],said:3,sake:3,same:[2,3],sampl:[1,2],satisfi:3,save:[1,2,3],saxexcept:2,scale:[2,4],screen:5,search:[0,2,3,4,5],searchabl:2,searchfield:5,sec:[2,5],second:[1,2,3,5],section:[2,3,4,5],see:[1,2,5],seem:3,seen:[2,5],segment:2,select:5,self:5,send:2,sens:[2,3],separ:[3,5],sequenc:3,serv:2,server:1,servflag:2,servic:[2,5],servlet:2,servletsbaseurl:5,servstat:2,set:[1,2],setattribut:2,setfield:2,setformat:2,setignoreattributeord:2,setignorecom:2,setignorediffbetweentextandcdata:2,setignorewhitespac:2,setnormalizewhitespac:2,setocc:2,setqueri:2,setrow:2,settext:2,setxlink:2,setzen:5,sever:2,share:1,she:1,shortcom:2,shorter:2,should:[2,3,4],show:2,shown:[2,3,5],side:[2,5],similar:2,simpl:[1,2,3,5],simplest:2,simpli:2,simplic:3,simplif:2,simplifi:2,sinc:[2,5],sind:2,size:[1,2,3],skeleton_layout_templ:5,slr:2,small:2,snippet:2,softwar:[0,2,3,5],solr:[2,5],solrclient:2,solrdocumentlist:2,solrend:2,solrqueri:2,solrresult:2,solrstart:2,solut:[2,3],solv:2,some:[2,3,4,5],someth:2,somewhat:2,sort:[1,2],sortedbylengthmap:2,sortoutaffix:2,sourc:[2,4,5],space:[2,5],span:5,special:5,specif:[2,3,4,5],specifi:[1,2,4],speech:3,sprach:3,src:[2,3,4,5],stand:2,standard:[1,2,3,4,5],standardfootnot:3,start:[1,2,3,5],startswith:2,state:2,statist:[1,2],stem:[2,3],stemnumb:2,step:2,still:[1,2,5],stop:2,store:5,stream:1,string:[2,3,5],stronger:3,structur:[2,3,4,5],style:[3,5],stylesheet:5,subdirectori:4,subdivid:2,subject:3,subsec:[2,3,4,5],subsect:[3,5],subsequ:3,substanti:2,substitut:1,substr:2,substract:2,subtl:1,subword:3,succe:1,success:2,suf:2,suffic:2,suffix:[2,3,5],suffixallomorph:2,suffixbaseform:3,suffixenum:2,suffixmorphem:2,suffixnumb:2,suggest:[1,2,3],sum:2,superus:5,support:3,suppos:2,sustain:3,syntact:3,syntax:5,system:2,tabindex:5,tabl:1,tag:[1,2,5],tagcorpusservlet:2,taggen:5,tagmanu:2,tagservlet:2,tagset:3,tagurl:2,take:[1,2],taken:2,target:[2,3],task:[2,3,5],technolog:[2,3],tei:[2,3,4,5],teiexamp:3,templat:5,term:3,test:[1,2,5],text:[1,2,3,5],textfil:2,tha:2,than:[2,3],theenumi:2,thei:[1,2,3,4,5],them:[1,2],theoret:[1,3],theori:5,therefor:[2,3,5],thereof:3,thi:[1,2,3,5],thing:5,think:2,third:[2,5],though:2,three:[2,4],through:[1,2,3,5],throughout:2,thu:[1,2,5],time:[2,3],timecorpusbegin:2,timecorpusend:2,timecorrect:2,togeth:[1,2,5],toggl:5,token:2,tomcat:2,tool:1,tostr:2,transfer:2,transform:5,translat:5,treebank:3,treemap:2,truth:2,turn:2,two:[1,2,5],type:[2,3,5],typic:2,ueberschrift:5,ufara:2,ufe:2,ufer:2,ufera:2,uferr:2,uferra:2,ufor:2,ufora:2,ufr:2,ufra:2,ufyrra:2,unbound:3,und:2,under:2,underli:[1,2],understand:[2,5],unequ:2,unfortun:[2,3,5],uniqu:[2,5],unit:[1,2],unknown:2,unlik:2,untag:[1,2,5],until:2,updat:2,updateend:2,updatestart:2,upload:[2,5],upper:[4,5],url:[2,5],urlencod:5,urn:5,use:[2,3,5],used:[1,2,3,5],user:[1,2,3,5],usernam:[2,5],uses:[2,5],using:2,usual:[1,2],utf:[3,5],uuer:2,uuera:2,uvver:2,uvvor:2,valid:[2,5],valu:[1,2,5],variabl:[2,5],variant:2,vereinheitlichung:2,veri:[1,2,3],version:[2,3,5],vfere:2,via:[2,4,5],view:[0,3,4],visibl:2,visual:[1,5],vuer:2,vver:2,wai:2,want:5,war:2,wdtpe:2,wealth:3,web:[2,3,5],webapplicationbaseurl:5,webfrag:[2,5],webinterfac:2,webpag:[2,5],webservic:5,well:[2,3],wenn:2,were:[2,3,5],what:2,when:2,where:2,wherea:[2,3],whether:1,which:[1,2,3,4,5],who:[1,5],whose:2,wirklich:5,within:[1,2,5],without:2,word:[1,2,3,5],wordbegin:3,worddatamodel:[2,3],wordroot:2,wordtoken:2,wordtyp:[2,3],work:[1,2],workaround:2,workload:1,worteben:3,worth:[2,3],worttyp:2,would:2,wrd:2,write:2,writeal:2,writealldata:2,writecont:2,writedb:5,written:[1,2],wrong:2,www:[2,3,4,5],xalan:5,xed:[2,5],xlink:[2,5],xlinknamespac:2,xml:[2,3,4,5],xmldiff:2,xmln:[3,5],xmloutputt:2,xmlschema:3,xmlunit:2,xpath:[2,5],xpathex:2,xpathexpress:2,xpathfactori:2,xpexp:2,xpfac:2,xpfacti:2,xsd:3,xsi:3,xsl:5,xslt:5,yes:1,yet:[1,2,3,5],yfera:2,yfere:2,yferra:2,zero:2,zerobegin:2,zeroend:2,zipf:2,zur:2},titles:["Documentation Morphilo Project","Software Design","Controller Adjustments","Data Model","Framework","View"],titleterms:{adjust:2,conceptu:[2,3,5],control:2,data:3,design:1,document:0,framework:4,gener:2,implement:[2,3,5],indic:0,model:3,morphilo:0,oper:2,principl:2,project:0,softwar:1,tabl:0,view:5,welcom:[]}})
\ No newline at end of file
+Search.setIndex({docnames:["index","source/architecture","source/controller","source/datamodel","source/view"],envversion:52,filenames:["index.rst","source/architecture.rst","source/controller.rst","source/datamodel.rst","source/view.rst"],objects:{},objnames:{},objtypes:{},terms:{"0px":4,"10px":4,"1test":4,"2test":4,"4em":4,"93ff":1,"\u00f6gai":1,"abstract":3,"boolean":2,"case":[1,2,3,4],"catch":2,"class":[1,2,4],"default":[2,4],"enum":2,"final":[2,3,4],"function":[2,3],"import":[1,2,3,4],"int":2,"kr\u00fcger":1,"new":[1,2],"null":2,"public":2,"return":2,"short":4,"throw":[2,3],"true":[1,2,3,4],"try":2,"void":2,"while":[1,2],AND:2,And:[1,2],For:[1,2,3,4],That:2,The:[1,2,3,4],Then:[2,4],There:[1,2,3],These:[1,2,3],Using:2,With:2,_morphilo_:4,abbmycorestruktur:[],abbruchbedingung:2,abl:[3,4],about:2,abov:[1,2,3],accept:2,access:[1,2,4],accessdelet:4,accessedit:4,accord:[2,3],accordingli:2,account:3,acl:4,actual:[1,2],add:2,added:[1,2,3],adding:2,addit:[2,3],addition:3,addon:[1,2],address:2,addtomorphilodb:2,adject:3,adjust:[0,1,3,4],administ:1,administr:[2,4],admit:2,adopt:2,advantag:3,affect:1,affix:[1,2,3],affixpo:[],affixstripp:2,affixtyp:2,after:[1,2,4],again:[2,4],agre:2,aim:2,al1:2,al2:2,algorithm:2,all:[1,2,3,4],allomorph:2,allow:[1,2,3,4],alon:2,alpha:[],alreadi:[1,2],also:[1,2,3,4],altern:1,although:[2,3],alwai:2,amadeusgwin:1,ambigu:3,among:4,amp:4,analys:2,analysi:[],analyz:[1,2,3],analyzeinfl:[],analyzeinflect:2,analyzelemma:2,analyzepref:[],analyzeprefix:2,analyzeroot:2,analyzesuf:[],analyzesuffix:2,analyzeword:2,andrea:1,ani:[1,2],annot:[1,2,4],annotieren:4,anoth:[1,2,4],anpasssungen:4,answer:3,antonio:1,anzword:2,apach:[2,4],api:2,appear:[1,2,4],appli:[2,4],applic:[1,2,3],approach:[1,4],appropri:3,approxim:2,arbitrari:2,architect:[],architectur:[0,3],argument:2,aria:4,around:2,arrai:2,arraylist:2,asid:3,assert:2,assign:2,assum:[1,2],assumpt:[1,3],ation:2,attach:[3,4],attent:[3,4],attribut:[2,3],auf:3,aur:2,auslesen:2,authen:2,authentif:2,author:2,autom:1,automat:[1,4],auver:2,avail:[1,2,3],avoid:2,awai:2,awkward:2,awver:2,back:[2,4],backtoproject:4,bar:4,base:[1,2,3,4],baseform:3,basi:2,basic:[1,2,3],becaus:[1,2,3,4],becom:[1,2],been:[2,3],befor:[1,2],beforehand:1,begin:[2,3],beginnig:2,begintim:2,behind:2,being:2,belong:4,below:[2,4],besid:4,best:3,between:[1,2,3,4],beyond:[],bird:2,bit:2,block:[],both:[1,2,3],branch:[1,3],brows:4,browser:[1,4],btn:4,bug:1,bugfixend:[],bugfixstart:[],build:[1,2],builder:2,button:4,butz:1,calcul:[1,2],call:[1,2,4],callkeymeth:[],calll:2,can:[1,2,3,4],candid:2,cannot:[2,3],caption:[],care:2,caret:4,carri:[1,2,3,4],caus:[2,3],center:[],certain:4,chang:[1,2,3,4],changeabl:2,chap:[],chapter:2,check:[2,4],checkand:2,checkpermiss:4,child:4,children:4,choic:3,chosen:3,circumst:4,claim:2,classdiag:[],classid:2,classif:3,classifi:3,clear:[2,4],clearer:1,click:2,client:[1,2],cmp:2,code:[1,2,4],cog:4,collect:[2,4],com:[1,3],come:2,comfort:3,common:4,commun:1,compar:[1,2],comparemcr:[],comparemcrobject:2,compareto:2,comparison:[1,2],compat:3,compil:2,complet:[1,2,3],complex:2,complextyp:3,complic:2,compon:[1,2,4],componentsprotectfootnotemark:[],compound:2,compris:[2,3],con:3,concat:4,concaten:2,concept:3,conceptu:0,concern:[1,2],conclud:2,concret:1,condit:2,configur:[1,2,4],confirm:2,confirm_delet:4,congruenc:1,consequ:[1,3],consid:2,consist:2,construct:[2,3],constructor:2,constructqc:[],contain:[1,2,4],content:[0,1,2,4],continu:1,control:[0,1,4],converg:3,core:2,corner:1,corp1derivid:2,corp2derivid:2,corpdoc:2,corpid:[2,4],corpmeta:[2,3,4],corpo1:2,corpo2:2,corpobj:2,corpora:[1,2,3],corpu:[1,2,3,4],corpusdatamodel:[],corpusdiff:2,corpuslink:[2,3,4],corpuss:2,correct:[1,2,4],correl:2,correspond:[1,2,3,4],could:[1,2,3],count:[1,2,4],counter:2,coupl:[1,4],cours:[1,2],creat:[1,2,4],createdbi:2,createmorphiloobject:2,createxml:2,creator:2,crpid1:2,crpid2:2,crucial:[1,2,3],css:4,current:4,currentus:[2,4],custom:[1,2],cut:2,data:[0,1,2,4],databas:[1,2],datamodel:[1,2,3],datamodelimpl:[],dataset:[1,2],date:[2,3],datefrom:[2,3],datefromcorp:2,datenmodel:1,dateuntil:[2,3],dateuntilcorp:2,deal:2,decid:1,decis:1,def:[1,2,4],defin:[1,2,3,4],definit:[2,4],degre:2,delet:[2,3,4],deleteaffix:[],deletedb:4,delin:1,deliv:[1,2],delword:4,demonstr:[2,4],depend:2,depict:[2,3],depth:2,deriv:[1,2,3,4],derivateact:4,derivcorp:4,derivid:2,derobject:[2,4],derobjecttempl:[],describ:[1,2,3,4],deserv:4,design:[0,2,4],detail:[2,4],detaileddiff:2,determin:[1,2,4],develop:[1,3],deviat:1,diachron:3,diagram:[],diagramm:2,dictionari:2,diff:2,diffbegin:[],diffend:[],differ:[1,2,3,4],difflist:[],diffsum:[],dimens:3,direct:[2,3],directli:[1,3],directori:[1,2,4],discuss:[2,4],displai:[2,4],distribut:[1,2],disus:3,div:4,divid:1,divis:4,doc:[],document:[1,2,3,4],doe:[1,2],doi:[],domin:3,done:[1,2,3,4],dontknow:3,doubl:2,down:[2,4],drop:4,dropdown:4,dropdownmenu1:4,due:2,dynam:1,each:[2,3,4],earlier:2,easi:3,easili:[1,2],editword:4,effect:2,effici:[2,3],either:2,elem:2,element:[2,3],elig:3,elimin:2,elm:2,els:2,elsewher:3,eman:3,embed:2,embedaffix:[],embedded:2,emph:[],empir:1,empti:2,enabl:2,enclos:3,encod:[2,3,4],end:[2,3],endswith:2,endtim:2,engin:[2,3,4],english:[1,2,4],enough:1,enrich:4,ensur:4,ent:2,enter:[1,4],entir:[2,4],entitl:4,entri:[1,2],entrynum:2,enumer:2,enumpref:[],envisu:1,equal:[1,2],equalobject:2,equalocc2:[],equalocc:[],equival:2,err:2,error:[1,3],erzeugen:2,escapechar:[],escpeci:1,especi:1,essenc:2,essenti:[1,4],establish:3,estim:1,etc:2,euer:2,european:3,evalu:2,evaluatefirst:2,even:[1,3],event:[1,4],everi:2,everyth:1,exact:[1,2],exactli:[1,2,4],exampl:[1,2,3],except:2,excerpt:[],exchang:[2,3],exclud:4,exist:[1,2],expand:4,explain:[1,2],explan:[],explanatori:4,explic:1,explicit:2,explicitli:[2,3],exploit:4,exponenti:[],extend:[1,2],extent:3,extra:[1,2],extract:[1,2,3,4],extrem:2,eye:2,fact:[1,2,3],factori:2,fail:2,fall:2,fals:[2,3],felder:3,few:3,field:[2,3,4],fig:[],figur:[1,2],file:[1,2,3,4],filenam:[2,4],filepathproc:2,filesaveend:[],filesavestart:[],fill:2,filter:2,find:2,first:[1,2,4],five:2,fix:2,flag:2,flow:1,focu:2,follow:[1,2,3,4],footnot:[],footnotetext:[],form:[1,2,4],format:2,former:[2,4],fort:3,four:[2,3],fragment:[2,4],framework:[0,2,3,4],freeli:2,frequenc:[1,2],frequent:2,from:[1,2,3,4],fuer:2,fulfil:2,full:2,fulli:[1,2],further:[1,2],furthermor:[3,4],futur:3,gauss:[],gegeben:2,gener:[0,1,4],german:4,get:2,getaffixposit:2,getalldiffer:2,getallequalmcrobject:2,getattributevalu:2,getbaseurl:2,getcont:2,getcontentfromfil:2,getcontrolnodedetail:2,getcorp:[],getcorpusmetadata:2,getcurrentsess:2,getderiv:[],getderivatefilepath:2,getequalnumb:2,getequalobjectnumb:2,getfieldvalu:2,getfilenam:2,getid:2,getinst:2,getmaindocnam:2,getmorphem:2,getnamespac:2,getnumberofword:2,getnumfound:2,getoccurrencesfromequaltext:2,getpath:2,getprettyformat:2,getresult:2,getsolrcli:2,getstemnumb:2,getter:2,gettestnodedetail:2,gettext:2,getunknownword:2,getunknowword:[],geturl:[],geturlparamet:2,getuserid:2,getuserinform:2,getvalu:2,getxmlfromobject:2,getxpathloc:2,gist:1,github:[1,3],give:2,given:[1,2,3,4],gleich:2,glyphicon:4,good:4,gov:4,grant:[],greater:2,group:[1,4],grow:1,gruyter:1,guest:4,guidelin:3,had:4,hamburg:1,hand:[1,2,3,4],handl:[1,2,4],happen:[2,4],harbor:[2,4],has:[1,2,3,4],hasderiv:3,hash:2,hashmap:2,hauver:2,have:[1,2,3,4],head:4,heavili:2,help:1,helpobj:2,henc:2,her:2,here:[1,2,3,4],herit:3,hidden:[1,4],hier:4,hierarch:3,hierarchi:[2,3],higher:2,his:2,histor:3,historyd:3,hit:2,horizont:4,houyr:2,hover:2,how:[1,2],howev:[1,2,3],href:[2,4],html:4,http:[1,2,3,4],httpsession:4,hundr:2,hypertext:4,i18n:4,idea:2,ideal:3,ident:2,identifi:[2,3,4],ids:[2,4],ifs:4,ifsdirectori:4,ignor:[2,3],illustr:[1,2],imag:4,impact:1,implement:[0,1],impli:[1,2,3,4],implicitli:2,importantli:2,improv:2,includ:[2,3,4],includegraph:[],increas:[],increment:2,incrocc:2,indent:4,independ:1,index:[0,1,2],indic:2,indo:3,infl:2,inflect:[2,3],inflectionenum:2,inflenum:2,inform:[1,3,4],ingredi:2,inherit:4,initi:[1,2,4],inner:2,input:1,inspir:2,instanc:[2,3],instanti:2,instead:1,instruct:2,integ:2,integr:2,intend:[],interact:[1,2,3,4],interakt:1,interest:[2,3],interfac:[1,2,4],interrupt:1,investig:[2,3],involv:[1,2],ioexcept:2,ion:2,isauthor:2,ischild:3,isempti:2,isequ:2,ispar:3,issu:[3,4],ist:2,item:[1,2],itemlabel:[],iter:2,itm:[],its:[1,2,3],java:[1,2,4],javacod:[],javascript:4,jdm:2,jdom:2,jdomdoc:2,jdomdochelp:2,jdomobject:[],jdomorphilo:2,job:2,just:[1,2,3],keep:[2,3],kei:[2,4],kept:3,keyboard:4,keyset:2,kind:3,known:3,korpu:4,korpusnam:[2,3],label:4,labelenumi:[],labelledbi:4,laid:[],landscap:[],languag:[1,3,4],larg:[1,2],larger:2,last:[2,3],lastli:[],later:[1,2,3],latter:[2,4],law:2,layer:[1,3],layout:[1,4],lead:3,least:2,left:[2,3,4],leftov:2,leftsov:2,lemma:[2,3,4],lemmaanalyz:[],length:[2,4],less:[2,3],let:[],letter:2,level:[1,2,3],lexic:[1,2,3],like:[1,2],limit:2,line:[2,3,4],linear:2,link:[1,2,3,4],list:[1,2,3,4],littl:3,loc:4,locat:4,loeschen:4,logic:[1,2,4],login:[],longer:2,look:[1,2],loop:[2,4],low:1,lower:2,lst:[],lstlist:2,machin:[1,4],made:[1,2],mai:1,main:[1,2,4],mainfil:2,mainli:2,major:3,make:[1,2,3,4],manag:[1,2,3],mani:[2,4],manipul:[2,4],manual:[1,2,4],map:[1,2],margin:4,mark:[1,2],markup:3,maschin:1,mask:[1,2,4],master:[1,2,4],match:[1,2,4],materi:1,matter:2,maxoccur:[2,3],mcr:4,mcr_directori:4,mcraccessmanag:4,mcridstr:2,mcrmetadatamanag:2,mcrmetalinkid:4,mcrobj1:2,mcrobj2:2,mcrobj:2,mcrobject:2,mcrobjectid:2,mcrobjekt:2,mcrpath:2,mcrservlet:2,mcrservletjob:2,mcrsessionmgr:2,mcrsolrclientfactori:2,mcrtranslat:4,mcrurn:4,mcrxmlfunction:[2,4],mcrxsl:4,mdm:2,mean:[1,2,3],measur:2,mechan:[2,4],meet:3,mensch:1,ment:2,mention:[2,3,4],menu:[2,4],menuitem:4,mere:2,messages_d:4,messages_en:4,met:4,meta:[2,3],metadata:[1,2,3,4],method:[1,2],middl:[1,2],might:[3,4],mileston:2,minoccur:3,mix:3,mod:4,mode:[1,2,4],model:[0,2,4],modif:4,modul:[0,3],modular:[1,2],monomorphem:2,more:[1,2,3,4],moreov:2,morphem:[2,3],morphil:[],morphilo:[2,3,4],morphilo_uml:[],morphilocontain:[2,3,4],morphilostylesheet:[],morphmenu:4,morpholog:[1,2],morphologicalsystem:[],most:[1,2,3,4],mous:4,multi:3,multilingu:4,multipl:2,must:[2,4],mvc:[0,3,4],mycor:[1,2,3,4],mycore_architectur:[],mycoreobject:[2,4],name:[2,3,4],namespac:2,natur:1,navig:4,necessari:[1,2,3,4],need:[1,2,3,4],neg:3,neither:2,net:4,nevertheless:1,newli:[1,2],newroot:2,next:[2,4],nextobject:4,nirgend:2,node:[2,3,4],nonamespaceschemaloc:3,nor:2,normal:[1,4],notavail:[],note:[1,2,4],notic:[3,4],notinherit:3,now:[2,3],number:[1,2,3],numberformatexcept:2,nur:2,nw1:[],nw2:[],oar:2,obaer:2,ober:2,obj:2,objactiontempl:[],object:[1,2,3,4],objectact:4,objectid:2,objecttyp:3,objid:[2,4],obligatori:3,observ:1,obuh:2,obviou:[],occdiff:2,occur:[1,2],occurr:[2,3],oder:2,oed:[],oedfootnot:[],oer:2,ofaer:2,ofer:2,oferr:2,off:2,offer:[1,2],offerr:2,offr:2,ofir:2,ofor:2,ofowr:2,oger:2,oher:2,onc:[2,4],one:[1,2,3,4],ones:2,ongo:1,onli:[1,2,3,4],onward:2,ooer:2,oor:2,oouer:2,open:4,oper:0,oppos:2,optim:1,option:[2,3,4],optional:3,order:2,org:[2,3,4],origin:2,other:[1,2,3,4],otherwis:[1,2,3],ouer:2,ouir:2,our:2,out:[1,2,3,4],outer:2,output:[1,2],outputstr:2,outputt:2,ouuer:2,ouur:2,ouver:2,ouyr:2,ova:2,ovah:2,ovar:2,ove:2,over:[1,2],overal:4,overrid:2,overwhelm:3,overwrit:2,overwritten:2,ovir:2,ovr:2,ovuh:2,ovur:2,ovver:2,ovyr:2,ower:2,owir:2,own:[1,4],owr:2,owuer:2,owur:2,owver:2,owwer:2,owyr:2,oxford:2,packag:2,page:[0,2,4],paid:3,par:3,parallel:1,param:4,paramet:[2,4],parencit:[],parentobjid:4,parseint:2,part:[2,3,4],partial:1,pass:2,path:[2,4],pattern:[1,2,3,4],pcs2:2,pcs:2,pdf:[],penn:3,persist:[1,3],perspect:2,peukert2012:[],peukert:1,pflichtfeld:3,phase:2,phonolog:3,place:[1,2],plain:1,plu:2,png:[],point:[1,2,4],popul:1,pos:[2,3],posit:[1,2,3],possibl:[1,2,3],post:2,potenti:[2,3],practic:[1,4],precis:[1,2],predefin:1,predict:3,pref:2,prefcutoff:[],prefenum:2,prefer:2,prefix:[1,2,3,4],prefixallomorph:2,prefixbaseform:3,prefixenum:2,prefixmorphem:2,prefixnumb:2,prefloop:[],prefputmorph:[],present:[1,2,3,4],prevent:2,previou:[1,2],previous:[1,2],principl:0,printstacktrac:2,privat:2,probabl:2,problem:[2,3],problemat:3,proc:2,procedur:[2,4],process:[1,2,3,4],processcorpusservlet:[2,4],processor:4,procwd:2,product:[2,3],program:[1,2],progress:1,project:[2,4],promot:1,prop:2,properti:[3,4],propos:2,protect:2,prototyp:[2,3],provid:[1,2,3],publish:[2,4],pull:4,put:[1,2],putal:2,qry:2,qualiti:[1,2],qualitycontrol:2,quantif:3,queri:2,question:[2,3],rang:2,rather:2,reach:[],read:[2,3,4],reader:4,readi:2,realiz:3,realli:[2,4],reason:[1,2,3],receiv:[1,2,4],recht:2,recogn:[1,2,3,4],recognit:2,recommend:4,recurs:2,redirect:2,ref:2,refactor:2,reflect:3,refln:[],regard:3,region:1,regist:[2,4],reject:1,rekurs:2,rel:4,relat:2,releas:[],relev:[2,3,4],reliabl:3,remain:2,remaind:[1,2,4],remark:3,remov:2,removecont:[],render:[2,4],renewcommand:[],repeat:2,repetit:2,replac:2,repositori:[0,2,3,4],represent:[2,4],request:2,requir:[1,3],research:[1,3],resolv:[2,4],resourc:[1,4],respect:[1,2,3,4],respond:2,respons:[2,4],rest:[1,2],restart:2,restrict:4,restword:2,result:[1,2,3,4],resultset:2,resum:[1,2],retrievemcrobject:2,reus:1,reveal:2,revers:2,right:[1,2,3,4],rise:[],role:[2,4],roll:2,root:[1,2,3,4],rootanalyz:[],roughli:1,row:4,rslt:2,rudimentari:2,run:[1,2,4],said:3,sake:3,same:[1,2,3],sampl:[1,2],satisfi:3,save:[1,2,3],saxexcept:2,scale:[],scientif:1,screen:[1,4],search:[0,1,2,3,4],searchabl:2,searchfield:4,sec:[],second:[1,2,3,4],section:[1,4],see:[1,2,4],seem:3,seen:[2,4],segment:2,select:4,self:4,semi:1,send:2,sens:[2,3],separ:[3,4],sequenc:3,seri:1,serv:[],server:1,servflag:2,servic:[2,4],servlet:2,servletsbaseurl:4,servstat:2,set:[1,2],setattribut:2,setfield:2,setformat:2,setignoreattributeord:2,setignorecom:2,setignorediffbetweentextandcdata:2,setignorewhitespac:2,setnormalizewhitespac:2,setocc:2,setqueri:2,setrow:2,settext:2,setxlink:2,setzen:4,sever:[1,2],share:1,she:1,shortcom:[],shorter:2,should:[1,2,3],show:2,shown:[2,4],side:[2,4],similar:2,simpl:[1,2,3,4],simplest:2,simpli:2,simplic:3,simplif:2,simplifi:2,simultan:1,sinc:[1,2,4],sind:2,size:[1,2,3],skeleton_layout_templ:4,slr:2,small:2,snippet:2,softwar:[0,2,3,4],solr:[2,4],solrclient:2,solrdocumentlist:2,solrend:[],solrqueri:2,solrresult:[],solrstart:[],solut:[2,3],solv:2,some:[1,2,3,4],someth:2,somewhat:2,sort:[1,2],sortedbylengthmap:2,sortoutaffix:2,sourc:[1,2,4],sourcecod:[],space:[2,4],span:4,special:4,specif:[1,2,3,4],specifi:[1,2],speech:3,sprach:3,src:[1,2,4],stand:2,standard:[1,2,3,4],standardfootnot:[],standardsn:3,start:[1,2,3,4],startswith:2,state:2,statist:[1,2],stem:[2,3],stemnumb:2,step:2,still:[1,2,4],stop:2,store:4,stream:[],string:[1,2,3,4],stronger:3,structur:[1,2,3,4],style:[3,4],stylesheet:4,subdirectori:1,subdivid:2,subject:3,subsec:[],subsect:4,subsequ:3,substanti:[1,2],substitut:1,substr:2,substract:2,subtl:1,subword:3,succe:1,success:2,suf:2,suffic:2,suffix:[1,2,3,4],suffixallomorph:2,suffixbaseform:3,suffixenum:2,suffixmorphem:2,suffixnumb:2,suggest:[1,2,3],sum:2,superus:4,support:[1,3],suppos:2,sustain:[1,3],syntact:3,syntax:4,system:[1,2],tabindex:4,tabl:1,tag:[1,2,4],tagcorpusservlet:2,taggen:4,tagmanu:[],tagservlet:[],tagset:3,tagurl:2,take:[1,2],taken:2,target:[2,3],task:[2,3,4],technolog:[2,3],tei:[2,3,4],teiexamp:[],templat:4,term:3,test:[1,2,4],text:[1,2,3,4],textfil:2,tha:2,than:[2,3],theenumi:[],thei:[1,2,3,4],them:[1,2],theoret:[1,3],theori:4,therefor:[2,3,4],thereof:3,thi:[1,2,3,4],thing:4,think:2,third:2,though:2,three:[1,2],threshold:1,through:[1,2,3,4],throughout:2,thu:[1,2,4],time:[1,2,3],timecorpusbegin:2,timecorpusend:2,timecorrect:2,togeth:[1,2,4],toggl:4,token:2,tomcat:2,tool:1,tostr:2,touch:1,transfer:2,transform:4,translat:4,treebank:3,treemap:2,truth:2,turn:2,two:[1,2,4],type:[2,3,4],typic:2,ueberschrift:4,ufara:2,ufe:2,ufer:2,ufera:2,uferr:2,uferra:2,ufor:2,ufora:2,ufr:2,ufra:2,ufyrra:2,uml:2,unbound:3,und:2,under:2,underli:[1,2],understand:[2,4],unequ:2,unfortun:[2,3,4],uni:1,uniform:1,uniqu:[2,4],unit:[1,2],unknown:2,unlik:2,untag:[1,2,4],until:2,updat:[1,2],updateend:[],updatestart:[],upload:[2,4],upper:[1,4],url:[2,4],urlencod:4,urn:4,use:[1,2,3,4],used:[1,2,3,4],user:[1,2,3,4],usernam:[2,4],uses:[1,2,4],using:[1,2],usual:[1,2],utf:[3,4],uuer:2,uuera:2,uvver:2,uvvor:2,valid:[2,4],valu:[1,2,4],variabl:[2,4],variant:2,vereinheitlichung:2,veri:[1,2,3],version:[2,3,4],vfere:2,via:[1,2,4],view:[0,1,3],visibl:2,visual:[1,4],vuer:2,vver:2,wai:2,want:4,war:2,wdtpe:2,wealth:3,web:[1,2,3,4],webapplicationbaseurl:4,webfrag:[],webinterfac:2,webpag:[2,4],webservic:4,webxml:[],well:[1,2,3],wenn:2,were:[2,3,4],what:2,when:2,where:2,wherea:[1,2,3],whether:1,which:[1,2,3,4],who:[1,4],whose:2,wien:1,wirklich:4,within:[1,2,4],without:[1,2],word:[1,2,3,4],wordbegin:[],worddatamodel:[],wordroot:2,wordtoken:2,wordtyp:[2,3],work:[1,2],workaround:2,workload:1,worteben:3,worth:[2,3],worttyp:2,would:2,wrd:2,write:2,writeal:2,writealldata:2,writecont:[],writedb:4,written:[1,2],wrong:[],www:[1,2,3,4],xalan:4,xed:[2,4],xlink:[2,4],xlinknamespac:2,xml:[1,2,3,4],xmldiff:2,xmln:[3,4],xmloutputt:2,xmlschema:3,xmlunit:2,xpath:[2,4],xpathex:2,xpathexpress:2,xpathfactori:2,xpexp:2,xpfac:2,xpfacti:2,xsd:3,xsi:3,xsl:4,xslt:4,yes:1,yet:[1,2,3,4],yfera:2,yfere:2,yferra:2,zero:2,zerobegin:[],zeroend:[],zipf:2,zur:2},titles:["Morphilo Project Documentation","Software Design","Controller Adjustments","Data Model","View"],titleterms:{adjust:2,architectur:1,basic:[],conceptu:[2,3,4],control:2,data:3,design:1,document:0,framework:1,gener:2,implement:[2,3,4],indic:0,model:[1,3],morphilo:[0,1],mvc:1,oper:2,principl:2,project:0,repositori:1,softwar:1,tabl:0,view:4,welcom:[]}})
\ No newline at end of file
diff --git a/Morphilo_doc/_build/html/source/architecture.html b/Morphilo_doc/_build/html/source/architecture.html
index bb49762dfb34c0975d8a375948adbcaa059fe235..efee7fd5cd22eb82f95c2eebf8d858ecd88fbdbb 100644
--- a/Morphilo_doc/_build/html/source/architecture.html
+++ b/Morphilo_doc/_build/html/source/architecture.html
@@ -15,8 +15,8 @@
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="Framework" href="framework.html" />
- <link rel="prev" title="View" href="view.html" />
+ <link rel="next" title="Data Model" href="datamodel.html" />
+ <link rel="prev" title="Morphilo Project Documentation" href="../index.html" />
<link rel="stylesheet" href="../_static/custom.css" type="text/css" />
@@ -33,50 +33,85 @@
<div class="section" id="software-design">
<h1>Software Design<a class="headerlink" href="#software-design" title="Permalink to this headline">¶</a></h1>
-<img alt="source/architecture.*" src="source/architecture.*" />
-<p>The architecture of a possible <strong>take-and-share</strong>-approach for language
-resources is visualized in figure ref{fig:architect}. Because the very gist
+<div class="section" id="mvc-model">
+<h2>MVC Model<a class="headerlink" href="#mvc-model" title="Permalink to this headline">¶</a></h2>
+<p>A standard architecture for software has become a form of an
+observer pattern called <em>Model-View-Controller (MVC)</em>-Model <a class="footnote-reference" href="#f3" id="id1">[3]</a>.
+This is escpecially true for web-based applications that use
+some form of a client-server architecture since these systems naturally divide
+the browser view from the rest of the program logic and, if dynamically set up,
+also from the data model usually running in an extra server as well.
+As already implied, the MVC-pattern modularizes the program into three components: model, view, and
+controller coupled <em>low</em> by interfaces. The view is concerned with
+everything the actual user sees on the screen or uses to interact with the
+machine. The controller is to recognize and process the events initiated by the
+user and to update the view. Processing involves to communicate with the model.
+This may involve to save or provide data from the data base.</p>
+<p>From all that follows, MVC-models are especially supportive for reusing
+existing software and promotes parallel development of its three components.
+So the data model of an existing program can easily be changed without touching
+the essentials of the program logic. The same is true for the code that handles
+the view. Most of the time view and data model are the two components that need
+to be changed so that the software appearance and presentation is adjusted to
+the new user group as well as the different data is adjusted to the needs of the different
+requirements of the new application. Nevertheless, if bugs or general changes in
+the controller component have to be done, it usually does not affect
+substantially the view and data model.</p>
+<p>Another positive consequence of MVC-models is that several views (or even
+models) could be used simultaneously. It means that the same data could be
+presented differently on the user interface.</p>
+</div>
+<div class="section" id="morphilo-architecture">
+<h2>Morphilo Architecture<a class="headerlink" href="#morphilo-architecture" title="Permalink to this headline">¶</a></h2>
+<div class="figure" id="id4">
+<img alt="../_images/architecture.png" src="../_images/architecture.png" />
+<p class="caption"><span class="caption-text">Figure 1: Basic Architecture of a Take-&-Share-Approach</span></p>
+</div>
+<p>The architecture of a possible <em>take-and-share</em> approach for language
+resources is visualized in figure 1. Because the very gist
of the approach becomes clearer if describing a concrete example, the case of
-annotating lexical derivatives of Middle English and a respective database is
-given as an illustration.
+annotating lexical derivatives of Middle English with the help of the Morphilo Tool
+<a class="footnote-reference" href="#f1" id="id2">[1]</a> using a <a class="reference external" href="http://www.mycore.de">MyCoRe repository</a> is given as an illustration.
However, any other tool that helps with manual annotations and manages metadata of a corpus could be
-substituted here instead.</p>
+substituted here instead. <a class="footnote-reference" href="#f2" id="id3">[2]</a></p>
<p>After inputting an untagged corpus or plain text, it is determined whether the
input material was annotated previously by a different user. This information is
usually provided by the metadata administered by the annotation tool; in the case at
-hand it is called emph{Morphilizer} in figure ref{fig:architect}. An
-alternative is a simple table look-up for all occurring words in the datasets Corpus 1 through Corpus n. If contained
-completely, the emph{yes}-branch is followed up further – otherwise emph{no}
+hand, the <em>Morphilo</em> component. An alternative is a
+simple table look-up for all occurring words in the datasets Corpus 1 through Corpus n. If contained
+completely, the <em>yes</em>-branch is followed up further – otherwise <em>no</em>
succeeds. The difference between the two branches is subtle, yet crucial. On
-both branches, the annotation tool (here emph{Morphilizer}) is called, which, first,
-sorts out all words that are not contained in the master database (here emph{Morphilo-DB})
+both branches, the annotation tool (here <em>Morphilo</em>) is called, which, first,
+sorts out all words that are not contained in the master database (here <em>MyCoRe</em> repository)
and, second, makes reasonable suggestions on an optimal annotation of
-the items. In both cases the
-annotations are linked to the respective items (e.g. words) in the
-text, but they are also persistently saved in an extra dataset, i.e. Corpus 1
+the items. The suggestions made to the user are based on simple string mapping of a saved list of prefixes and suffixes
+whereas the remainder of the mapping is defined as the word root. The annotations are linked to the respective items (e.g. words) in the
+text, but they are also persistently saved in an extra dataset, i.e. in figure 1 in one of the delineated Corpus 1
through n, together with all available metadata.</p>
-<p>The difference between both information streams is that
-in the emph{yes}-branch a comparison between the newly created dataset and
-all of the previous datasets of this text is carried out. Within this
-unit, all deviations and congruencies are marked and counted. The underlying
+<p>The difference between the two branches in figure 1 is that
+in the <em>yes</em>-branch a comparison between the newly created dataset and
+all of the previous datasets of this text is carried out while this is not
+possible if a text was not annotated before. Within this
+unit, all deviations and congruencies of the annotated items are marked and counted. The underlying
assumption is that with a growing number of comparable texts the
correct annotations approach a theoretic true value of a correct annotation
while errors level out provided that the sample size is large enough. How the
distribution of errors and correct annotations exactly looks like and if a
normal distribution can be assumed is still object of the ongoing research, but
-independent of the concrete results, the component (called emph{compare
-manual annotations} in figure ref{fig:architect}) allows for specifying the
+independent of the concrete results, the component (called <em>compare
+manual annotations</em> in figure 1) allows for specifying the
exact form of the sample population.
In fact, it is necessary at that point to define the form of the distribution,
-sample size, and the rejection region. The standard setting are a normal
-distribution, a rejection region of $alpha = 0.05$ and sample size of $30$ so
-that a simple Gauss-Test can be calculated.</p>
-<p>Continuing the information flow further, these statistical calculations are
+sample size, and the rejection region. To be put it simple here, a uniform distribution in form of a threshold value
+of e.g. 20 could be defined that specifies that a word has to be annotated equally by
+20 different users before it enters the master database.</p>
+<p>Continuing the information flow in figure 1 further, the threshold values or, if so defined,
+the results of the statistical calculation of other distributions respectively are
delivered to the quality-control-component. Based on the statistics, the
respective items together with the metadata, frequencies, and, of course,
annotations are written to the master database. All information in the master
database is directly used for automated annotations. Thus it is directly matched
-to the input texts or corpora respectively through the emph{Morphilizer}-tool.
+to the input texts or corpora respectively through the <em>Morphilo</em>-tool.
The annotation tool decides on the entries looked up in the master which items
are to be manually annotated.</p>
<p>The processes just described are all hidden from the user who has no possibility
@@ -89,6 +124,48 @@ the user will have access to the annotations made in the respective dataset,
correct them or save them and resume later. It is important to note that the user will receive
the tagged document only after all items are fully annotated. No partially
tagged text can be output.</p>
+</div>
+<div class="section" id="repository-framework">
+<h2>Repository Framework<a class="headerlink" href="#repository-framework" title="Permalink to this headline">¶</a></h2>
+<div class="figure" id="id5">
+<img alt="../_images/mycore_architecture-2.png" src="../_images/mycore_architecture-2.png" />
+<p class="caption"><span class="caption-text">Figure 2: <a class="reference external" href="http://www.mycore.de">MyCoRe</a>-Architecture and Components</span></p>
+</div>
+<p>To specify the repository framework, the morphilo application logic will have to be implemented,
+a data model specified, and the input, search and output mask programmed.</p>
+<p>There are three directories which are
+important for adjusting the MyCoRe framework to the needs of one’s own application.</p>
+<p>These three directories
+correspond essentially to the three components in the MVC model as explicated above. Roughly, they are also envisualized in figure 2 in the upper
+right hand corner. More precisely, the view (<em>Layout</em> in figure 2) and the model layer
+(<em>Datenmodell</em> in figure 2) can be done
+completely via the <em>interface</em>, which is a directory with a predefined
+structure and some standard files. For the configuration of the logic an extra directory is offered (<em>/src/main/java/custom/mycore/addons/</em>). Here all, java classes
+extending the controller layer should be added.
+Practically, all three MVC layers are placed in the
+<em>src/main/</em>-directory of the application. In one of the subdirectories,
+<em>datamodel/def,</em> the datamodel specifications are defined as xml files. It parallels the model
+layer in the MVC pattern. How the data model was defined will be explained in the section Data Model.</p>
+<p class="rubric">Notes</p>
+<table class="docutils footnote" frame="void" id="f1" rules="none">
+<colgroup><col class="label" /><col /></colgroup>
+<tbody valign="top">
+<tr><td class="label"><a class="fn-backref" href="#id2">[1]</a></td><td>Peukert, H. (2012): From Semi-Automatic to Automatic Affix Extraction in Middle English Corpora: Building a Sustainable Database for Analyzing Derivational Morphology over Time, Empirical Methods in Natural Language Processing, Wien, Scientific series of the ÖGAI, 413-23.</td></tr>
+</tbody>
+</table>
+<table class="docutils footnote" frame="void" id="f2" rules="none">
+<colgroup><col class="label" /><col /></colgroup>
+<tbody valign="top">
+<tr><td class="label"><a class="fn-backref" href="#id3">[2]</a></td><td>The source code of a possible implementation is available on <a class="reference external" href="https://github.com/amadeusgwin/morphilo">https://github.com/amadeusgwin/morphilo</a>. The software runs in test mode on <a class="reference external" href="https://www.morphilo.uni-hamburg.de/content/index.xml">https://www.morphilo.uni-hamburg.de/content/index.xml</a>.</td></tr>
+</tbody>
+</table>
+<table class="docutils footnote" frame="void" id="f3" rules="none">
+<colgroup><col class="label" /><col /></colgroup>
+<tbody valign="top">
+<tr><td class="label"><a class="fn-backref" href="#id1">[3]</a></td><td>Butz, Andreas; Antonio Krüger (2017): Mensch-Maschine-Interaktion, De Gruyter, 93ff.</td></tr>
+</tbody>
+</table>
+</div>
</div>
@@ -96,12 +173,22 @@ tagged text can be output.</p>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
- <div class="sphinxsidebarwrapper"><div class="relations">
+ <div class="sphinxsidebarwrapper">
+ <h3><a href="../index.html">Table Of Contents</a></h3>
+ <ul>
+<li><a class="reference internal" href="#">Software Design</a><ul>
+<li><a class="reference internal" href="#mvc-model">MVC Model</a></li>
+<li><a class="reference internal" href="#morphilo-architecture">Morphilo Architecture</a></li>
+<li><a class="reference internal" href="#repository-framework">Repository Framework</a></li>
+</ul>
+</li>
+</ul>
+<div class="relations">
<h3>Related Topics</h3>
<ul>
<li><a href="../index.html">Documentation overview</a><ul>
- <li>Previous: <a href="view.html" title="previous chapter">View</a></li>
- <li>Next: <a href="framework.html" title="next chapter">Framework</a></li>
+ <li>Previous: <a href="../index.html" title="previous chapter">Morphilo Project Documentation</a></li>
+ <li>Next: <a href="datamodel.html" title="next chapter">Data Model</a></li>
</ul></li>
</ul>
</div>
diff --git a/Morphilo_doc/_build/html/source/architecture.pdf b/Morphilo_doc/_build/html/source/architecture.pdf
deleted file mode 100644
index 2f671b5abd23245db733040b2e56873976a2f9fd..0000000000000000000000000000000000000000
Binary files a/Morphilo_doc/_build/html/source/architecture.pdf and /dev/null differ
diff --git a/Morphilo_doc/_build/html/source/controller.html b/Morphilo_doc/_build/html/source/controller.html
index 92abe3a963110abf67f18cc5dcfc31a372e2b587..94c20a4aa70ce058f493eb5873483dab8bca9a5f 100644
--- a/Morphilo_doc/_build/html/source/controller.html
+++ b/Morphilo_doc/_build/html/source/controller.html
@@ -15,8 +15,7 @@
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="View" href="view.html" />
- <link rel="prev" title="Data Model" href="datamodel.html" />
+ <link rel="prev" title="View" href="view.html" />
<link rel="stylesheet" href="../_static/custom.css" type="text/css" />
@@ -35,51 +34,43 @@
<h1>Controller Adjustments<a class="headerlink" href="#controller-adjustments" title="Permalink to this headline">¶</a></h1>
<div class="section" id="general-principle-of-operation">
<h2>General Principle of Operation<a class="headerlink" href="#general-principle-of-operation" title="Permalink to this headline">¶</a></h2>
-<p>Figure ref{fig:classDiag} illustrates the dependencies of the five java classes that were integrated to add the morphilo
-functionality defined in the default package emph{custom.mycore.addons.morphilo}. The general principle of operation
+<div class="figure" id="id2">
+<span id="classdiag"></span><img alt="../_images/morphilo_uml.png" src="../_images/morphilo_uml.png" />
+<p class="caption"><span class="caption-text">Figure 3: Morphilo UML Diagramm</span></p>
+</div>
+<p>Figure <a class="reference internal" href="#classdiag"><span class="std std-ref">Figure 3: Morphilo UML Diagramm</span></a> illustrates the dependencies of the five java classes that were integrated to add the morphilo
+functionality defined in the default package <em>custom.mycore.addons.morphilo</em>. The general principle of operation
is the following. The handling of data search, upload, saving, and user
authentification is fully left to the MyCoRe functionality that is completely
-implemented. The class emph{ProcessCorpusServlet.java} receives a request from the webinterface to process an uploaded file,
+implemented. The class <em>ProcessCorpusServlet.java</em> receives a request from the webinterface to process an uploaded file,
i.e. a simple text corpus, and it checks if any of the words are available in the master database. All words that are not
listed in the master database are written to an extra file. These are the words that have to be manually annotated. At the end, the
servlet sends a response back to the user interface. In case of all words are contained in the master, an xml file is generated from the
master database that includes all annotated words of the original corpus. Usually this will not be the case for larger textfiles.
So if some words are not in the master, the user will get the response to initiate the manual annotation process.</p>
<p>The manual annotation process is processed by the class
-emph{{Tag-Corpus-Serv-let-.ja-va}}, which will build a JDOM object for the first word in the extra file.
-This is done by creating an object of the emph{JDOMorphilo.java} class. This class, in turn, will use the methods of
-emph{AffixStripper.java} that make simple, but reasonable, suggestions on the word structure. This JDOM object is then
+<em>TagCorpusServlet.java</em>, which will build a JDOM object for the first word in the extra file.
+This is done by creating an object of the <em>JDOMorphilo.java</em> class. This class, in turn, will use the methods of
+<em>AffixStripper.java</em> that make simple, but reasonable, suggestions on the word structure. This JDOM object is then
given as a response back to the user. It is presented as a form, in which the user can make changes. This is necessary
-because the word structure algorithm of emph{AffixStripper.java} errs in some cases. Once the user agrees on the
+because the word structure algorithm of <em>AffixStripper.java</em> errs in some cases. Once the user agrees on the
suggestions or on his or her corrections, the JDOM object is saved as an xml that is only searchable, visible, and
changeable by the authenicated user (and the administrator), another file containing all processed words is created or
-updated respectively and the emph{TagCorpusServlet.java} servlet will restart until the last word in the extra list is
+updated respectively and the <em>TagCorpusServlet.java</em> servlet will restart until the last word in the extra list is
processed. This enables the user to stop and resume her or his annotation work at a later point in time. The
-emph{TagCorpusServlet} will call methods from emph{ProcessCorpusServlet.java} to adjust the content of the extra
+<em>TagCorpusServlet</em> will call methods from <em>ProcessCorpusServlet.java</em> to adjust the content of the extra
files harboring the untagged words. If this file is empty, and only then, it is replaced by the file comprising all words
from the original text file, both the ones from the master database and the ones that are annotated by the user,
in an annotated xml representation.</p>
-<p>Each time emph{ProcessCorpusServlet.java} is instantiated, it also instantiates emph{QualityControl.java}. This class checks if a
+<p>Each time <em>ProcessCorpusServlet.java</em> is instantiated, it also instantiates <em>QualityControl.java</em>. This class checks if a
new word can be transferred to the master database. The algorithm can be freely adopted to higher or lower quality standards.
In its present configuration, a method tests at a limit of 20 different
registered users agreeing on the annotation of the same word. More specifically,
-if 20 JDOM objects are identical except in the attribute field emph{occurrences} in the metadata node, the JDOM object becomes
-part of the master. The latter is easily done by changing the attribute emph{creator} from the user name
-to emph{<a href="#id1"><span class="problematic" id="id2">``</span></a>administrator’‘} in the service node. This makes the dataset part of the master database. Moreover, the emph{occurrences}
+if 20 JDOM objects are identical except in the attribute field <em>occurrences</em> in the metadata node, the JDOM object becomes
+part of the master. The latter is easily done by changing the attribute <em>creator</em> from the user name
+to <em>administrator</em> in the service node. This makes the dataset part of the master database. Moreover, the <em>occurrences</em>
attribute is updated by adding up all occurrences of the word that stem from
-different text corpora of the same time range.
-begin{landscape}</p>
-<blockquote>
-<div><dl class="docutils">
-<dt>begin{figure}</dt>
-<dd>centering
-includegraphics[scale=0.55]{morphilo_uml.png}
-caption{Class Diagram Morphilo}
-label{fig:classDiag}</dd>
-</dl>
-<p>end{figure}</p>
-</div></blockquote>
-<p>end{landscape}</p>
+different text corpora of the same time range.</p>
</div>
<div class="section" id="conceptualization">
<h2>Conceptualization<a class="headerlink" href="#conceptualization" title="Permalink to this headline">¶</a></h2>
@@ -90,914 +81,1165 @@ files as saving, creating, deleting, and updating files.
Moreover, a rudimentary user management comprising different roles and
rights is offered. The basic technology behind the controller’s logic is the
servlet. As such all new code has to be registered as a servlet in the
-web-fragment.xml (here the Apache Tomcat container) as listing ref{lst:webfragment} shows.</p>
-<p>begin{lstlisting}[language=XML,caption={Servlet Registering in the
-web-fragment.xml (excerpt)},label=lst:webfragment,escapechar=|]
-<servlet></p>
-<blockquote>
-<div><servlet-name>ProcessCorpusServlet</servlet-name>
-<servlet-class>custom.mycore.addons.morphilo.ProcessCorpusServlet</servlet-class></div></blockquote>
-<p></servlet>
-<servlet-mapping></p>
-<blockquote>
-<div><servlet-name>ProcessCorpusServlet</servlet-name>
-<url-pattern>/servlets/object/process</url-pattern>|label{ln:process}|</div></blockquote>
-<p></servlet-mapping>
-<servlet></p>
-<blockquote>
-<div><servlet-name>TagCorpusServlet</servlet-name>
-<servlet-class>custom.mycore.addons.morphilo.TagCorpusServlet</servlet-class></div></blockquote>
-<p></servlet>
-<servlet-mapping></p>
-<blockquote>
-<div><servlet-name>TagCorpusServlet</servlet-name>
-<url-pattern>/servlets/object/tag</url-pattern>|label{ln:tag}|</div></blockquote>
-<p></servlet-mapping>
-end{lstlisting}</p>
-<p>Now, the logic has to be extended by the specifications analyzed in chapter
-ref{chap:concept} on conceptualization. More specifically, some
+web-fragment.xml (here the Apache Tomcat container).</p>
+<div class="literal-block-wrapper docutils container" id="id3">
+<span id="webxml"></span><div class="code-block-caption"><span class="caption-text">Servlet Registering in the web-fragment.xml</span><a class="headerlink" href="#id3" title="Permalink to this code">¶</a></div>
+<div class="highlight-xml notranslate"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+10
+11
+12
+13
+14
+15
+16
+17</pre></div></td><td class="code"><div class="highlight"><pre><span></span><span class="nt"><servlet></span>
+ <span class="nt"><servlet-name></span>ProcessCorpusServlet<span class="nt"></servlet-name></span>
+ <span class="nt"><servlet-class></span>custom.mycore.addons.morphilo.ProcessCorpusServlet<span class="nt"></servlet-class></span>
+<span class="nt"></servlet></span>
+<span class="nt"><servlet-mapping></span>
+ <span class="nt"><servlet-name></span>ProcessCorpusServlet<span class="nt"></servlet-name></span>
+<span class="hll"> <span class="nt"><url-pattern></span>/servlets/object/process<span class="nt"></url-pattern></span>
+</span><span class="nt"></servlet-mapping></span>
+<span class="nt"><servlet></span>
+ <span class="nt"><servlet-name></span>TagCorpusServlet<span class="nt"></servlet-name></span>
+ <span class="nt"><servlet-class></span>custom.mycore.addons.morphilo.TagCorpusServlet<span class="nt"></servlet-class></span>
+<span class="nt"></servlet></span>
+<span class="nt"><servlet-mapping></span>
+ <span class="nt"><servlet-name></span>TagCorpusServlet<span class="nt"></servlet-name></span>
+<span class="hll"> <span class="nt"><url-pattern></span>/servlets/object/tag<span class="nt"></url-pattern></span>
+</span><span class="nt"></servlet-mapping></span>
+\end{lstlisting}
+</pre></div>
+</td></tr></table></div>
+</div>
+<p>Now, the logic has to be extended by the specifications. Some
classes have to be added that take care of analyzing words
-(emph{AffixStripper.java, InflectionEnum.java, SuffixEnum.java,
-PrefixEnum.java}), extracting the relevant words from the text and checking the
-uniqueness of the text (emph{ProcessCorpusServlet.java}), make reasonable
-suggestions on the annotation (emph{TagCorpusServlet.java}), build the object
-of each annotated word (emph{JDOMorphilo.java}), and check on the quality by applying
-statistical models (emph{QualityControl.java}).</p>
+(<em>AffixStripper.java, InflectionEnum.java, SuffixEnum.java,
+PrefixEnum.java</em>), extracting the relevant words from the text and checking the
+uniqueness of the text (<em>ProcessCorpusServlet.java</em>), make reasonable
+suggestions on the annotation (<em>TagCorpusServlet.java</em>), build the object
+of each annotated word (<em>JDOMorphilo.java</em>), and check on the quality by applying
+statistical models (<em>QualityControl.java</em>).</p>
</div>
<div class="section" id="implementation">
-<h2>Implementation<a class="headerlink" href="#implementation" title="Permalink to this headline">¶</a></h2>
+<span id="controller-section"></span><h2>Implementation<a class="headerlink" href="#implementation" title="Permalink to this headline">¶</a></h2>
<p>Having taken a bird’s eye perspective in the previous chapter, it is now time to take a look at the specific implementation at the level
-of methods. Starting with the main servlet, emph{ProcessCorpusServlet.java}, the class defines four getter method:
-renewcommand{labelenumi}{(theenumi)}
-begin{enumerate}</p>
-<blockquote>
-<div>itemlabel{itm:geturl} public String getURLParameter(MCRServletJob, String)
-itemlabel{itm:getcorp} public String getCorpusMetadata(MCRServletJob, String)
-itemlabel{itm:getcont} public ArrayList<String> getContentFromFile(MCRServletJob, String)
-itemlabel{itm:getderiv} public Path getDerivateFilePath(MCRServletJob, String)
-itemlabel{itm:now} public int getNumberOfWords(MCRServletJob job, String)</div></blockquote>
-<p>end{enumerate}
-Since each servlet in MyCoRe extends the class MCRServlet, it has access to MCRServletJob, from which the http requests and responses
-can be used. This is the first argument in the above methods. The second argument of method (ref{itm:geturl}) specifies the name of an url parameter, i.e.
-the object id or the id of the derivate. The method returns the value of the given parameter. Typically MyCoRe uses the url to exchange
+of methods. Starting with the main servlet, <em>ProcessCorpusServlet.java</em>, the class defines four getter method:</p>
+<ol class="arabic simple">
+<li><em>public String getURLParameter(MCRServletJob, String)</em></li>
+<li><em>public String getCorpusMetadata(MCRServletJob, String)</em></li>
+<li><em>public ArrayList<String> getContentFromFile(MCRServletJob, String)</em></li>
+<li><em>public Path getDerivateFilePath(MCRServletJob, String)</em></li>
+<li><em>public int getNumberOfWords(MCRServletJob job, String)</em></li>
+</ol>
+<p>Since each servlet in MyCoRe extends the class MCRServlet, it has access to MCRServletJob, from which the http requests and responses
+can be used. This is the first argument in the above methods. The second argument of method (in 1.) specifies the name of an url parameter, i.e.
+the object id or the id of the derivate. The method returns the value of the given parameter. Typically <em>MyCoRe</em> uses the url to exchange
these ids. The second method provides us with the value of a data field in the xml document. So the string defines the name of an attribute.
-emph{getContentFromFile(MCRServletJob, String)} returns the words as a list from a file when given the filename as a string.
-The getter listed in ref{itm:getderiv}), returns the Path from the MyCoRe repository when the name of
-the file is specified. And finally, method (ref{itm:now}) returns the number of words by simply returning
-emph{getContentFromFile(job, fileName).size()}.</p>
+<em>getContentFromFile(MCRServletJob, String)</em> returns the words as a list from a file when given the filename as a string.
+The getter listed in 4., returns the Path from the MyCoRe repository when the name of
+the file is specified. And finally, method (in 5.) returns the number of words by simply returning
+<em>getContentFromFile(job, fileName).size()</em>.</p>
<p>There are two methods in every MyCoRe-Servlet that have to be overwritten,
-emph{protected void render(MCRServletJob, Exception)}, which redirects the requests as emph{POST} or emph{GET} responds, and
-emph{protected void think(MCRServletJob)}, in which the logic is implemented. Since the latter is important to understand the
-core idea of the Morphilo algorithm, it is displayed in full length in source code ref{src:think}.</p>
-<p>begin{lstlisting}[language=java,caption={The overwritten think method},label=src:think,escapechar=|]
-protected void think(MCRServletJob job) throws Exception
-{</p>
-<blockquote>
-<div><p>this.job = job;
-String dateFromCorp = getCorpusMetadata(job, “def.datefrom”);
-String dateUntilCorp = getCorpusMetadata(job, “def.dateuntil”);
-String corpID = getURLParameter(job, “objID”);
-String derivID = getURLParameter(job, “id”);</p>
-<p>//if NoW is 0, fill with anzWords
-MCRObject helpObj = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(corpID));|label{ln:bugfixstart}|
-Document jdomDocHelp = helpObj.createXML();
-XPathFactory xpfacty = XPathFactory.instance();
-XPathExpression<Element> xpExp = xpfacty.compile(“//NoW”, Filters.element());
-Element elem = xpExp.evaluateFirst(jdomDocHelp);
-//fixes transferred morphilo data from previous stand alone project
-int corpussize = getNumberOfWords(job, “”);
-if (Integer.parseInt(elem.getText()) != corpussize)
-{</p>
-<blockquote>
-<div>elem.setText(Integer.toString(corpussize));
-helpObj = new MCRObject(jdomDocHelp);
-MCRMetadataManager.update(helpObj);</div></blockquote>
-<p>}|label{ln:bugfixend}|</p>
-<p>//Check if the uploaded corpus was processed before
-SolrClient slr = MCRSolrClientFactory.getSolrClient();|label{ln:solrstart}|
-SolrQuery qry = new SolrQuery();
-qry.setFields(“korpusname”, “datefrom”, “dateuntil”, “NoW”, “id”);
-qry.setQuery(“datefrom:” + dateFromCorp + ” AND dateuntil:” + dateUntilCorp + ” AND NoW:” + corpussize);
-SolrDocumentList rslt = slr.query(qry).getResults();|label{ln:solrresult}|</p>
-<p>Boolean incrOcc = true;
-// if resultset contains only one, then it must be the newly created corpus
-if (slr.query(qry).getResults().getNumFound() > 1)
-{</p>
-<blockquote>
-<div>incrOcc = false;</div></blockquote>
-<p>}|label{ln:solrend}|</p>
-<p>//match all words in corpus with morphilo (creator=administrator) and save all words that are not in morphilo DB in leftovers
-ArrayList<String> leftovers = new ArrayList<String>();
-ArrayList<String> processed = new ArrayList<String>();</p>
-<p>leftovers = getUnknownWords(getContentFromFile(job, “”), dateFromCorp, dateUntilCorp, “”, incrOcc, incrOcc, false);|label{ln:callkeymeth}|</p>
-<p>//write all words of leftover in file as derivative to respective corpmeta dataset
-MCRPath root = MCRPath.getPath(derivID, “/”);|label{ln:filesavestart}|
-Path fn = getDerivateFilePath(job, “”).getFileName();
-Path p = root.resolve(“untagged-” + fn);
-Files.write(p, leftovers);|label{ln:filesaveend}|</p>
-<p>//create a file for all words that were processed
-Path procWds = root.resolve(“processed-” + fn);
-Files.write(procWds, processed);</p>
-</div></blockquote>
-<p>}
-end{lstlisting}
-Using the above mentioned getter methods, the emph{think} method assigns values to the object ID, needed to get the xml document
-that contain the corpus metadata, the file ID, and the beginning and starting dates from the corpus to be analyzed. Lines ref{ln:bugfixstart}
-through ref{ln:bugfixend} show how to access a mycore object as an xml document, a procedure that will be used in different variants
+<em>protected void render(MCRServletJob, Exception)</em>, which redirects the requests as <em>POST</em> or <em>GET</em> responds, and
+<em>protected void think(MCRServletJob)</em>, in which the logic is implemented. Since the latter is important to understand the
+core idea of the Morphilo algorithm, it is displayed in full length in source code <a class="reference internal" href="#think"><span class="std std-ref">The overwritten think method</span></a>.</p>
+<div class="literal-block-wrapper docutils container" id="id4">
+<span id="think"></span><div class="code-block-caption"><span class="caption-text">The overwritten think method</span><a class="headerlink" href="#id4" title="Permalink to this code">¶</a></div>
+<div class="highlight-java notranslate"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+10
+11
+12
+13
+14
+15
+16
+17
+18
+19
+20
+21
+22
+23
+24
+25
+26
+27
+28
+29
+30
+31
+32
+33
+34
+35
+36
+37
+38
+39
+40
+41
+42
+43
+44
+45
+46
+47
+48
+49
+50
+51
+52
+53</pre></div></td><td class="code"><div class="highlight"><pre><span></span><span class="kd">protected</span> <span class="kt">void</span> <span class="nf">think</span><span class="o">(</span><span class="n">MCRServletJob</span> <span class="n">job</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span>
+<span class="o">{</span>
+ <span class="k">this</span><span class="o">.</span><span class="na">job</span> <span class="o">=</span> <span class="n">job</span><span class="o">;</span>
+ <span class="n">String</span> <span class="n">dateFromCorp</span> <span class="o">=</span> <span class="n">getCorpusMetadata</span><span class="o">(</span><span class="n">job</span><span class="o">,</span> <span class="s">"def.datefrom"</span><span class="o">);</span>
+ <span class="n">String</span> <span class="n">dateUntilCorp</span> <span class="o">=</span> <span class="n">getCorpusMetadata</span><span class="o">(</span><span class="n">job</span><span class="o">,</span> <span class="s">"def.dateuntil"</span><span class="o">);</span>
+ <span class="n">String</span> <span class="n">corpID</span> <span class="o">=</span> <span class="n">getURLParameter</span><span class="o">(</span><span class="n">job</span><span class="o">,</span> <span class="s">"objID"</span><span class="o">);</span>
+ <span class="n">String</span> <span class="n">derivID</span> <span class="o">=</span> <span class="n">getURLParameter</span><span class="o">(</span><span class="n">job</span><span class="o">,</span> <span class="s">"id"</span><span class="o">);</span>
+
+ <span class="c1">//if NoW is 0, fill with anzWords</span>
+<span class="hll"> <span class="n">MCRObject</span> <span class="n">helpObj</span> <span class="o">=</span> <span class="n">MCRMetadataManager</span><span class="o">.</span><span class="na">retrieveMCRObject</span><span class="o">(</span><span class="n">MCRObjectID</span><span class="o">.</span><span class="na">getInstance</span><span class="o">(</span><span class="n">corpID</span><span class="o">));</span>
+</span> <span class="n">Document</span> <span class="n">jdomDocHelp</span> <span class="o">=</span> <span class="n">helpObj</span><span class="o">.</span><span class="na">createXML</span><span class="o">();</span>
+ <span class="n">XPathFactory</span> <span class="n">xpfacty</span> <span class="o">=</span> <span class="n">XPathFactory</span><span class="o">.</span><span class="na">instance</span><span class="o">();</span>
+ <span class="n">XPathExpression</span><span class="o"><</span><span class="n">Element</span><span class="o">></span> <span class="n">xpExp</span> <span class="o">=</span> <span class="n">xpfacty</span><span class="o">.</span><span class="na">compile</span><span class="o">(</span><span class="s">"//NoW"</span><span class="o">,</span> <span class="n">Filters</span><span class="o">.</span><span class="na">element</span><span class="o">());</span>
+ <span class="n">Element</span> <span class="n">elem</span> <span class="o">=</span> <span class="n">xpExp</span><span class="o">.</span><span class="na">evaluateFirst</span><span class="o">(</span><span class="n">jdomDocHelp</span><span class="o">);</span>
+ <span class="c1">//fixes transferred morphilo data from previous stand alone project</span>
+ <span class="kt">int</span> <span class="n">corpussize</span> <span class="o">=</span> <span class="n">getNumberOfWords</span><span class="o">(</span><span class="n">job</span><span class="o">,</span> <span class="s">""</span><span class="o">);</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">Integer</span><span class="o">.</span><span class="na">parseInt</span><span class="o">(</span><span class="n">elem</span><span class="o">.</span><span class="na">getText</span><span class="o">())</span> <span class="o">!=</span> <span class="n">corpussize</span><span class="o">)</span>
+ <span class="o">{</span>
+ <span class="n">elem</span><span class="o">.</span><span class="na">setText</span><span class="o">(</span><span class="n">Integer</span><span class="o">.</span><span class="na">toString</span><span class="o">(</span><span class="n">corpussize</span><span class="o">));</span>
+ <span class="n">helpObj</span> <span class="o">=</span> <span class="k">new</span> <span class="n">MCRObject</span><span class="o">(</span><span class="n">jdomDocHelp</span><span class="o">);</span>
+ <span class="n">MCRMetadataManager</span><span class="o">.</span><span class="na">update</span><span class="o">(</span><span class="n">helpObj</span><span class="o">);</span>
+<span class="hll"> <span class="o">}</span>
+</span>
+ <span class="c1">//Check if the uploaded corpus was processed before</span>
+<span class="hll"> <span class="n">SolrClient</span> <span class="n">slr</span> <span class="o">=</span> <span class="n">MCRSolrClientFactory</span><span class="o">.</span><span class="na">getSolrClient</span><span class="o">();</span>
+</span> <span class="n">SolrQuery</span> <span class="n">qry</span> <span class="o">=</span> <span class="k">new</span> <span class="n">SolrQuery</span><span class="o">();</span>
+ <span class="n">qry</span><span class="o">.</span><span class="na">setFields</span><span class="o">(</span><span class="s">"korpusname"</span><span class="o">,</span> <span class="s">"datefrom"</span><span class="o">,</span> <span class="s">"dateuntil"</span><span class="o">,</span> <span class="s">"NoW"</span><span class="o">,</span> <span class="s">"id"</span><span class="o">);</span>
+ <span class="n">qry</span><span class="o">.</span><span class="na">setQuery</span><span class="o">(</span><span class="s">"datefrom:"</span> <span class="o">+</span> <span class="n">dateFromCorp</span> <span class="o">+</span> <span class="s">" AND dateuntil:"</span> <span class="o">+</span> <span class="n">dateUntilCorp</span> <span class="o">+</span> <span class="s">" AND NoW:"</span> <span class="o">+</span> <span class="n">corpussize</span><span class="o">);</span>
+ <span class="n">SolrDocumentList</span> <span class="n">rslt</span> <span class="o">=</span> <span class="n">slr</span><span class="o">.</span><span class="na">query</span><span class="o">(</span><span class="n">qry</span><span class="o">).</span><span class="na">getResults</span><span class="o">();</span>
+
+ <span class="n">Boolean</span> <span class="n">incrOcc</span> <span class="o">=</span> <span class="kc">true</span><span class="o">;</span>
+ <span class="c1">// if resultset contains only one, then it must be the newly created corpus</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">slr</span><span class="o">.</span><span class="na">query</span><span class="o">(</span><span class="n">qry</span><span class="o">).</span><span class="na">getResults</span><span class="o">().</span><span class="na">getNumFound</span><span class="o">()</span> <span class="o">></span> <span class="mi">1</span><span class="o">)</span>
+ <span class="o">{</span>
+ <span class="n">incrOcc</span> <span class="o">=</span> <span class="kc">false</span><span class="o">;</span>
+<span class="hll"> <span class="o">}</span>
+</span>
+ <span class="c1">//match all words in corpus with morphilo (creator=administrator) and save all words that are not in morphilo DB in leftovers</span>
+ <span class="n">ArrayList</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">leftovers</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ArrayList</span><span class="o"><</span><span class="n">String</span><span class="o">>();</span>
+ <span class="n">ArrayList</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">processed</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ArrayList</span><span class="o"><</span><span class="n">String</span><span class="o">>();</span>
+
+<span class="hll"> <span class="n">leftovers</span> <span class="o">=</span> <span class="n">getUnknownWords</span><span class="o">(</span><span class="n">getContentFromFile</span><span class="o">(</span><span class="n">job</span><span class="o">,</span> <span class="s">""</span><span class="o">),</span> <span class="n">dateFromCorp</span><span class="o">,</span> <span class="n">dateUntilCorp</span><span class="o">,</span> <span class="s">""</span><span class="o">,</span> <span class="n">incrOcc</span><span class="o">,</span> <span class="n">incrOcc</span><span class="o">,</span> <span class="kc">false</span><span class="o">);</span>
+</span>
+ <span class="c1">//write all words of leftover in file as derivative to respective corpmeta dataset</span>
+<span class="hll"> <span class="n">MCRPath</span> <span class="n">root</span> <span class="o">=</span> <span class="n">MCRPath</span><span class="o">.</span><span class="na">getPath</span><span class="o">(</span><span class="n">derivID</span><span class="o">,</span> <span class="s">"/"</span><span class="o">);</span>
+</span> <span class="n">Path</span> <span class="n">fn</span> <span class="o">=</span> <span class="n">getDerivateFilePath</span><span class="o">(</span><span class="n">job</span><span class="o">,</span> <span class="s">""</span><span class="o">).</span><span class="na">getFileName</span><span class="o">();</span>
+ <span class="n">Path</span> <span class="n">p</span> <span class="o">=</span> <span class="n">root</span><span class="o">.</span><span class="na">resolve</span><span class="o">(</span><span class="s">"untagged-"</span> <span class="o">+</span> <span class="n">fn</span><span class="o">);</span>
+<span class="hll"> <span class="n">Files</span><span class="o">.</span><span class="na">write</span><span class="o">(</span><span class="n">p</span><span class="o">,</span> <span class="n">leftovers</span><span class="o">);</span>
+</span>
+ <span class="c1">//create a file for all words that were processed</span>
+ <span class="n">Path</span> <span class="n">procWds</span> <span class="o">=</span> <span class="n">root</span><span class="o">.</span><span class="na">resolve</span><span class="o">(</span><span class="s">"processed-"</span> <span class="o">+</span> <span class="n">fn</span><span class="o">);</span>
+ <span class="n">Files</span><span class="o">.</span><span class="na">write</span><span class="o">(</span><span class="n">procWds</span><span class="o">,</span> <span class="n">processed</span><span class="o">);</span>
+<span class="o">}</span>
+</pre></div>
+</td></tr></table></div>
+</div>
+<p>Using the above mentioned getter methods, the <em>think</em>-method assigns values to the object ID, needed to get the xml document
+that contains the corpus metadata, the file ID, and the beginning and starting dates from the corpus to be analyzed. Lines 10
+through 22 show how to access a mycore object as an xml document, a procedure that will be used in different variants
throughout this implementation.
By means of the object ID, the respective corpus is identified and a JDOM document is constructed, which can then be accessed
by XPath. The XPath factory instances are collections of the xml nodes. In the present case, it is save to assume that only one element
-of emph{NoW} is available (see corpus datamodel listing ref{lst:corpusdatamodel} with $maxOccurs=‘1’$). So we do not have to loop through
-the collection, but use the first node named emph{NoW}. The if-test checks if the number of words of the uploaded file is the
+of <em>NoW</em> is available (see corpus datamodel listing <a class="reference internal" href="datamodel.html#corpusdatamodel"><span class="std std-ref">Corpus Data Model</span></a> with <em>maxOccurs=‘1’</em>). So we do not have to loop through
+the collection, but use the first node named <em>NoW</em>. The if-test checks if the number of words of the uploaded file is the
same as the number written in the document. When the document is initially created by the MyCoRe logic it was configured to be zero.
-If unequal, the setText(String) method is used to write the number of words of the corpus to the document.</p>
-<p>Lines ref{ln:solrstart}–ref{ln:solrend} reveal the second important ingredient, i.e. controlling the search engine. First, a solr
+If unequal, the <em>setText(String)</em> method is used to write the number of words of the corpus to the document.</p>
+<p>Lines 25–36 reveal the second important ingredient, i.e. controlling the search engine. First, a solr
client and a query are initialized. Then, the output of the result set is defined by giving the fields of interest of the document.
-In the case at hand, it is the id, the name of the corpus, the number of words, and the beginnig and ending dates. With emph{setQuery}
-it is possible to assign values to some or all of these fields. Finally, emph{getResults()} carries out the search and writes
-all hits to a emph{SolrDocumentList} (line ref{ln:solrresult}). The test that follows is really only to set a Boolean
+In the case at hand, it is the id, the name of the corpus, the number of words, and the beginnig and ending dates. With <em>setQuery</em>
+it is possible to assign values to some or all of these fields. Finally, <em>getResults()</em> carries out the search and writes
+all hits to a <em>SolrDocumentList</em> (line 29). The test that follows is really only to set a Boolean
encoding if the number of occurrences of that word in the master should be updated. To avoid multiple counts,
incrementing the word frequency is only done if it is a new corpus.</p>
-<p>In line ref{ln:callkeymeth} emph{getUnknownWords(ArrayList, String, String, String, Boolean, Boolean, Boolean)} is called and
+<p>In line 42 <em>getUnknownWords(ArrayList, String, String, String, Boolean, Boolean, Boolean)</em> is called and
returned as a list of words. This method is key and will be discussed in depth below. Finally, lines
-ref{ln:filesavestart}–ref{ln:filesaveend} show how to handle file objects in MyCoRe. Using the file ID, the root path and the name
-of the first file in that path are identified. Then, a second file starting with <a href="#id3"><span class="problematic" id="id4">``</span></a>untagged’’ is created and all words returned from
-the emph{getUnknownWords} is written to that file. By the same token an empty file is created (in the last two lines of the emph{think}-method),
+45–48 show how to handle file objects in MyCoRe. Using the file ID, the root path and the name
+of the first file in that path are identified. Then, a second file starting with <em>untagged</em> is created and all words returned from
+the <em>getUnknownWords</em> is written to that file. By the same token an empty file is created (in the last two lines of the <em>think</em>-method),
in which all words that are manually annotated will be saved.</p>
-<p>In a refactoring phase, the method emph{getUnknownWords(ArrayList, String, String, String, Boolean, Boolean, Boolean)} could be subdivided into
+<p>In a refactoring phase, the method <em>getUnknownWords(ArrayList, String, String, String, Boolean, Boolean, Boolean)</em> could be subdivided into
three methods: for each Boolean parameter one. In fact, this method handles more than one task. This is mainly due to multiple code avoidance.
-%this is just wrong because no resultset will substantially be more than 10-20
-%In addition, for large text files this method would run into efficiency problems if the master database also reaches the intended size of about
-%$100,000$ entries and beyond because
In essence, an outer loop runs through all words of the corpus and an inner loop runs through all hits in the solr result set. Because the result
-set is supposed to be small, approximately between $10-20$ items, efficiency
+set is supposed to be small, approximately between 10-20 items, efficiency
problems are unlikely to cause a problem, although there are some more loops running through collection of about the same sizes.
-%As the hits naturally grow larger with an increasing size of the data base, processing time will rise exponentially.
Since each word is identified on the basis of its projected word type, the word form, and the time range it falls into, it is these variables that
have to be checked for existence in the documents. If not in the xml documents,
-emph{null} is returned and needs to be corrected. Moreover, user authentification must be considered. There are three different XPaths that are relevant.
-begin{itemize}</p>
-<blockquote>
-<div>item[-] emph{//service/servflags/servflag[@type=’createdby’]} to test for the correct user
-item[-] emph{//morphiloContainer/morphilo} to create the annotated document
-item[-] emph{//morphiloContainer/morphilo/w} to set occurrences or add a link</div></blockquote>
-<p>end{itemize}</p>
-<p>As an illustration of the core functioning of this method, listing ref{src:getUnknowWords} is given.
-begin{lstlisting}[language=java,caption={Mode of Operation of getUnknownWords Method},label=src:getUnknowWords,escapechar=|]
-public ArrayList<String> getUnknownWords(</p>
-<blockquote>
-<div><p>ArrayList<String> corpus,
-String timeCorpusBegin,
-String timeCorpusEnd,
-String wdtpe,
-Boolean setOcc,
-Boolean setXlink,
-Boolean writeAllData) throws Exception
-{</p>
-<blockquote>
-<div><p>String currentUser = MCRSessionMgr.getCurrentSession().getUserInformation().getUserID();
-ArrayList lo = new ArrayList();</p>
-<p>for (int i = 0; i < corpus.size(); i++)
-{</p>
-<blockquote>
-<div><p>SolrClient solrClient = MCRSolrClientFactory.getSolrClient();
-SolrQuery query = new SolrQuery();
-query.setFields(“w”,”occurrence”,”begin”,”end”, “id”, “wordtype”);
-query.setQuery(corpus.get(i));
-query.setRows(50); //more than 50 items are extremely unlikely
-SolrDocumentList results = solrClient.query(query).getResults();
-Boolean available = false;
-for (int entryNum = 0; entryNum < results.size(); entryNum++)
-{</p>
-<blockquote>
-<div>…
-// update in MCRMetaDataManager
-String mcrIDString = results.get(entryNum).getFieldValue(“id”).toString();
-//MCRObjekt auslesen und JDOM-Document erzeugen:
-MCRObject mcrObj = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(mcrIDString));
-Document jdomDoc = mcrObj.createXML();
-…
-//check and correction for word type
-…
-//checkand correction time: timeCorrect
-…
-//check if user correct: isAuthorized</div></blockquote>
-<p>…
-XPathExpression<Element> xp = xpfac.compile(“//morphiloContainer/morphilo/w”, Filters.element());
-//Iterates w-elements and increments occurrence attribute if setOcc is true
-for (Element e : xp.evaluate(jdomDoc))
-{</p>
-<blockquote>
-<div><dl class="docutils">
-<dt>//wenn Rechte da sind und Worttyp nirgends gegeben oder gleich ist</dt>
-<dd><blockquote class="first">
-<div><dl class="docutils">
-<dt>if (isAuthorized && timeCorrect</dt>
-<dd>&& ((e.getAttributeValue(“wordtype”) == null && wdtpe.equals(“”))
-|| e.getAttributeValue(“wordtype”).equals(wordtype))) // nur zur Vereinheitlichung</dd>
-</dl>
-</div></blockquote>
-<dl class="last docutils">
-<dt>{</dt>
-<dd><blockquote class="first">
-<div>int oc = -1;
-available = true;|label{ln:available}|</div></blockquote>
-<dl class="last docutils">
-<dt>try</dt>
-<dd><blockquote class="first">
-<div><dl class="docutils">
-<dt>{</dt>
-<dd>//adjust occurrence Attribut
-if (setOcc)</dd>
-</dl>
-</div></blockquote>
-<dl class="last docutils">
-<dt>{</dt>
-<dd><dl class="first last docutils">
-<dt>oc = Integer.parseInt(e.getAttributeValue(“occurrence”));</dt>
-<dd><blockquote class="first">
-<div>e.setAttribute(“occurrence”, Integer.toString(oc + 1));</div></blockquote>
-<p class="last">}</p>
-</dd>
-</dl>
-</dd>
-<dt>//write morphilo-ObjectID in xml of corpmeta</dt>
-<dd><blockquote class="first">
-<div><blockquote>
-<div><blockquote>
-<div><p>if (setXlink)
-{</p>
-<blockquote>
-<div>Namespace xlinkNamespace = Namespace.getNamespace(“xlink”, “<a class="reference external" href="http://www.w3.org/1999/xlink">http://www.w3.org/1999/xlink</a>”);|label{ln:namespace}|
-MCRObject corpObj = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(getURLParameter(job, “objID”)));
-Document corpDoc = corpObj.createXML();
-XPathExpression<Element> xpathEx = xpfac.compile(“//corpuslink”, Filters.element());
-Element elm = xpathEx.evaluateFirst(corpDoc);
-elm.setAttribute(“href” , mcrIDString, xlinkNamespace);</div></blockquote>
-<p>}
-mcrObj = new MCRObject(jdomDoc);|label{ln:updatestart}|
-MCRMetadataManager.update(mcrObj);
-QualityControl qc = new QualityControl(mcrObj);|label{ln:updateend}|</p>
-</div></blockquote>
-<p>}
-catch(NumberFormatException except)
-{</p>
-<blockquote>
-<div>// ignore</div></blockquote>
-<p>}</p>
-</div></blockquote>
-<p>}</p>
-</div></blockquote>
-<p class="last">}</p>
-</dd>
-</dl>
-</dd>
-</dl>
-</dd>
-</dl>
-</dd>
-</dl>
-<p>if (!available) // if not available in datasets under the given conditions <a href="#id16"><span class="problematic" id="id17">|\label{ln:notavailable}|</span></a>
-{</p>
-<blockquote>
-<div>lo.add(corpus.get(i));</div></blockquote>
-<p>}</p>
-</div></blockquote>
-<p>}
-return lo;</p>
-</div></blockquote>
-<p>}</p>
-</div></blockquote>
-</div></blockquote>
-<p>end{lstlisting}
-As can be seen from the functionality of listing ref{src:getUnknowWords}, getting the unknown words of a corpus, is rather a side effect for the equally named method.
-More precisely, a Boolean (line ref{ln:available}) is set when the document is manipulated otherwise because it is clear that the word must exist then.
-If the Boolean remains false (line ref{ln:notavailable}), the word is put on the list of words that have to be annotated manually. As already explained above, the
+<em>null</em> is returned and needs to be corrected. Moreover, user authentification must be considered. There are three different XPaths that are relevant.</p>
+<ul class="simple">
+<li><em>//service/servflags/servflag[@type=’createdby’]</em> to test for the correct user</li>
+<li><em>//morphiloContainer/morphilo</em> to create the annotated document</li>
+<li><em>//morphiloContainer/morphilo/w</em> to set occurrences or add a link</li>
+</ul>
+<p>As an illustration of the core functioning of this method, listing <a class="reference internal" href="#getunknowwords"><span class="std std-ref">Mode of Operation of getUnknownWords Method</span></a> is given.</p>
+<div class="literal-block-wrapper docutils container" id="id5">
+<span id="getunknowwords"></span><div class="code-block-caption"><span class="caption-text">Mode of Operation of getUnknownWords Method</span><a class="headerlink" href="#id5" title="Permalink to this code">¶</a></div>
+<div class="highlight-java notranslate"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+10
+11
+12
+13
+14
+15
+16
+17
+18
+19
+20
+21
+22
+23
+24
+25
+26
+27
+28
+29
+30
+31
+32
+33
+34
+35
+36
+37
+38
+39
+40
+41
+42
+43
+44
+45
+46
+47
+48
+49
+50
+51
+52
+53
+54
+55
+56
+57
+58
+59
+60
+61
+62
+63
+64
+65
+66
+67
+68
+69
+70
+71
+72
+73
+74
+75
+76
+77
+78
+79
+80
+81
+82
+83</pre></div></td><td class="code"><div class="highlight"><pre><span></span><span class="kd">public</span> <span class="n">ArrayList</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="nf">getUnknownWords</span><span class="o">(</span>
+ <span class="n">ArrayList</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">corpus</span><span class="o">,</span>
+ <span class="n">String</span> <span class="n">timeCorpusBegin</span><span class="o">,</span>
+ <span class="n">String</span> <span class="n">timeCorpusEnd</span><span class="o">,</span>
+ <span class="n">String</span> <span class="n">wdtpe</span><span class="o">,</span>
+ <span class="n">Boolean</span> <span class="n">setOcc</span><span class="o">,</span>
+ <span class="n">Boolean</span> <span class="n">setXlink</span><span class="o">,</span>
+ <span class="n">Boolean</span> <span class="n">writeAllData</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span>
+ <span class="o">{</span>
+ <span class="n">String</span> <span class="n">currentUser</span> <span class="o">=</span> <span class="n">MCRSessionMgr</span><span class="o">.</span><span class="na">getCurrentSession</span><span class="o">().</span><span class="na">getUserInformation</span><span class="o">().</span><span class="na">getUserID</span><span class="o">();</span>
+ <span class="n">ArrayList</span> <span class="n">lo</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ArrayList</span><span class="o">();</span>
+
+ <span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span> <span class="n">i</span> <span class="o"><</span> <span class="n">corpus</span><span class="o">.</span><span class="na">size</span><span class="o">();</span> <span class="n">i</span><span class="o">++)</span>
+ <span class="o">{</span>
+ <span class="n">SolrClient</span> <span class="n">solrClient</span> <span class="o">=</span> <span class="n">MCRSolrClientFactory</span><span class="o">.</span><span class="na">getSolrClient</span><span class="o">();</span>
+ <span class="n">SolrQuery</span> <span class="n">query</span> <span class="o">=</span> <span class="k">new</span> <span class="n">SolrQuery</span><span class="o">();</span>
+ <span class="n">query</span><span class="o">.</span><span class="na">setFields</span><span class="o">(</span><span class="s">"w"</span><span class="o">,</span><span class="s">"occurrence"</span><span class="o">,</span><span class="s">"begin"</span><span class="o">,</span><span class="s">"end"</span><span class="o">,</span> <span class="s">"id"</span><span class="o">,</span> <span class="s">"wordtype"</span><span class="o">);</span>
+ <span class="n">query</span><span class="o">.</span><span class="na">setQuery</span><span class="o">(</span><span class="n">corpus</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="n">i</span><span class="o">));</span>
+ <span class="n">query</span><span class="o">.</span><span class="na">setRows</span><span class="o">(</span><span class="mi">50</span><span class="o">);</span> <span class="c1">//more than 50 items are extremely unlikely</span>
+ <span class="n">SolrDocumentList</span> <span class="n">results</span> <span class="o">=</span> <span class="n">solrClient</span><span class="o">.</span><span class="na">query</span><span class="o">(</span><span class="n">query</span><span class="o">).</span><span class="na">getResults</span><span class="o">();</span>
+ <span class="n">Boolean</span> <span class="n">available</span> <span class="o">=</span> <span class="kc">false</span><span class="o">;</span>
+ <span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">entryNum</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span> <span class="n">entryNum</span> <span class="o"><</span> <span class="n">results</span><span class="o">.</span><span class="na">size</span><span class="o">();</span> <span class="n">entryNum</span><span class="o">++)</span>
+ <span class="o">{</span>
+ <span class="o">...</span>
+ <span class="c1">// update in MCRMetaDataManager</span>
+ <span class="n">String</span> <span class="n">mcrIDString</span> <span class="o">=</span> <span class="n">results</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="n">entryNum</span><span class="o">).</span><span class="na">getFieldValue</span><span class="o">(</span><span class="s">"id"</span><span class="o">).</span><span class="na">toString</span><span class="o">();</span>
+ <span class="c1">//MCRObjekt auslesen und JDOM-Document erzeugen:</span>
+ <span class="n">MCRObject</span> <span class="n">mcrObj</span> <span class="o">=</span> <span class="n">MCRMetadataManager</span><span class="o">.</span><span class="na">retrieveMCRObject</span><span class="o">(</span><span class="n">MCRObjectID</span><span class="o">.</span><span class="na">getInstance</span><span class="o">(</span><span class="n">mcrIDString</span><span class="o">));</span>
+ <span class="n">Document</span> <span class="n">jdomDoc</span> <span class="o">=</span> <span class="n">mcrObj</span><span class="o">.</span><span class="na">createXML</span><span class="o">();</span>
+ <span class="o">...</span>
+ <span class="c1">//check and correction for word type</span>
+ <span class="o">...</span>
+ <span class="c1">//checkand correction time: timeCorrect</span>
+ <span class="o">...</span>
+ <span class="c1">//check if user correct: isAuthorized</span>
+ <span class="o">...</span>
+ <span class="n">XPathExpression</span><span class="o"><</span><span class="n">Element</span><span class="o">></span> <span class="n">xp</span> <span class="o">=</span> <span class="n">xpfac</span><span class="o">.</span><span class="na">compile</span><span class="o">(</span><span class="s">"//morphiloContainer/morphilo/w"</span><span class="o">,</span> <span class="n">Filters</span><span class="o">.</span><span class="na">element</span><span class="o">());</span>
+ <span class="c1">//Iterates w-elements and increments occurrence attribute if setOcc is true</span>
+ <span class="k">for</span> <span class="o">(</span><span class="n">Element</span> <span class="n">e</span> <span class="o">:</span> <span class="n">xp</span><span class="o">.</span><span class="na">evaluate</span><span class="o">(</span><span class="n">jdomDoc</span><span class="o">))</span>
+ <span class="o">{</span>
+ <span class="c1">//wenn Rechte da sind und Worttyp nirgends gegeben oder gleich ist</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">isAuthorized</span> <span class="o">&&</span> <span class="n">timeCorrect</span>
+ <span class="o">&&</span> <span class="o">((</span><span class="n">e</span><span class="o">.</span><span class="na">getAttributeValue</span><span class="o">(</span><span class="s">"wordtype"</span><span class="o">)</span> <span class="o">==</span> <span class="kc">null</span> <span class="o">&&</span> <span class="n">wdtpe</span><span class="o">.</span><span class="na">equals</span><span class="o">(</span><span class="s">""</span><span class="o">))</span>
+ <span class="o">||</span> <span class="n">e</span><span class="o">.</span><span class="na">getAttributeValue</span><span class="o">(</span><span class="s">"wordtype"</span><span class="o">).</span><span class="na">equals</span><span class="o">(</span><span class="n">wordtype</span><span class="o">)))</span> <span class="c1">// nur zur Vereinheitlichung</span>
+ <span class="o">{</span>
+ <span class="kt">int</span> <span class="n">oc</span> <span class="o">=</span> <span class="o">-</span><span class="mi">1</span><span class="o">;</span>
+<span class="hll"> <span class="n">available</span> <span class="o">=</span> <span class="kc">true</span><span class="o">;</span>
+</span> <span class="k">try</span>
+ <span class="o">{</span>
+ <span class="c1">//adjust occurrence Attribut</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">setOcc</span><span class="o">)</span>
+ <span class="o">{</span>
+ <span class="n">oc</span> <span class="o">=</span> <span class="n">Integer</span><span class="o">.</span><span class="na">parseInt</span><span class="o">(</span><span class="n">e</span><span class="o">.</span><span class="na">getAttributeValue</span><span class="o">(</span><span class="s">"occurrence"</span><span class="o">));</span>
+ <span class="n">e</span><span class="o">.</span><span class="na">setAttribute</span><span class="o">(</span><span class="s">"occurrence"</span><span class="o">,</span> <span class="n">Integer</span><span class="o">.</span><span class="na">toString</span><span class="o">(</span><span class="n">oc</span> <span class="o">+</span> <span class="mi">1</span><span class="o">));</span>
+ <span class="o">}</span>
+
+ <span class="c1">//write morphilo-ObjectID in xml of corpmeta</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">setXlink</span><span class="o">)</span>
+ <span class="o">{</span>
+<span class="hll"> <span class="n">Namespace</span> <span class="n">xlinkNamespace</span> <span class="o">=</span> <span class="n">Namespace</span><span class="o">.</span><span class="na">getNamespace</span><span class="o">(</span><span class="s">"xlink"</span><span class="o">,</span> <span class="s">"http://www.w3.org/1999/xlink"</span><span class="o">);</span>
+</span> <span class="n">MCRObject</span> <span class="n">corpObj</span> <span class="o">=</span> <span class="n">MCRMetadataManager</span><span class="o">.</span><span class="na">retrieveMCRObject</span><span class="o">(</span><span class="n">MCRObjectID</span><span class="o">.</span><span class="na">getInstance</span><span class="o">(</span><span class="n">getURLParameter</span><span class="o">(</span><span class="n">job</span><span class="o">,</span> <span class="s">"objID"</span><span class="o">)));</span>
+ <span class="n">Document</span> <span class="n">corpDoc</span> <span class="o">=</span> <span class="n">corpObj</span><span class="o">.</span><span class="na">createXML</span><span class="o">();</span>
+ <span class="n">XPathExpression</span><span class="o"><</span><span class="n">Element</span><span class="o">></span> <span class="n">xpathEx</span> <span class="o">=</span> <span class="n">xpfac</span><span class="o">.</span><span class="na">compile</span><span class="o">(</span><span class="s">"//corpuslink"</span><span class="o">,</span> <span class="n">Filters</span><span class="o">.</span><span class="na">element</span><span class="o">());</span>
+ <span class="n">Element</span> <span class="n">elm</span> <span class="o">=</span> <span class="n">xpathEx</span><span class="o">.</span><span class="na">evaluateFirst</span><span class="o">(</span><span class="n">corpDoc</span><span class="o">);</span>
+ <span class="n">elm</span><span class="o">.</span><span class="na">setAttribute</span><span class="o">(</span><span class="s">"href"</span> <span class="o">,</span> <span class="n">mcrIDString</span><span class="o">,</span> <span class="n">xlinkNamespace</span><span class="o">);</span>
+ <span class="o">}</span>
+<span class="hll"> <span class="n">mcrObj</span> <span class="o">=</span> <span class="k">new</span> <span class="n">MCRObject</span><span class="o">(</span><span class="n">jdomDoc</span><span class="o">);</span>
+</span> <span class="n">MCRMetadataManager</span><span class="o">.</span><span class="na">update</span><span class="o">(</span><span class="n">mcrObj</span><span class="o">);</span>
+<span class="hll"> <span class="n">QualityControl</span> <span class="n">qc</span> <span class="o">=</span> <span class="k">new</span> <span class="n">QualityControl</span><span class="o">(</span><span class="n">mcrObj</span><span class="o">);</span>
+</span> <span class="o">}</span>
+ <span class="k">catch</span><span class="o">(</span><span class="n">NumberFormatException</span> <span class="n">except</span><span class="o">)</span>
+ <span class="o">{</span>
+ <span class="c1">// ignore</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+<span class="hll"> <span class="k">if</span> <span class="o">(!</span><span class="n">available</span><span class="o">)</span> <span class="c1">// if not available in datasets under the given conditions</span>
+</span> <span class="o">{</span>
+ <span class="n">lo</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="n">corpus</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="n">i</span><span class="o">));</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+ <span class="k">return</span> <span class="n">lo</span><span class="o">;</span>
+ <span class="o">}</span>
+</pre></div>
+</td></tr></table></div>
+</div>
+<p>As can be seen from the functionality of listing <a class="reference internal" href="#getunknowwords"><span class="std std-ref">Mode of Operation of getUnknownWords Method</span></a>, getting the unknown words of a corpus, is rather a side effect for the equally named method.
+More precisely, a Boolean (line 47) is set when the document is manipulated otherwise because it is clear that the word must exist then.
+If the Boolean remains false (line 77), the word is put on the list of words that have to be annotated manually. As already explained above, the
first loop runs through all words (corpus) and the following lines a solr result set is created. This set is also looped through and it is checked if the time range,
-the word type and the user are authorized. In the remainder, the occurrence attribute of the morphilo document can be incremented (setOcc is true) or/and the word is linked to the
-corpus meta data (setXlink is true). While all code lines are equivalent with
-what was explained in listing ref{src:think}, it suffices to focus on an
+the word type and the user are authorized. In the remainder, the occurrence attribute of the morphilo document can be incremented (<em>setOcc</em> is true) or/and the word is linked to the
+corpus meta data (<em>setXlink</em> is true). While all code lines are equivalent with
+what was explained in listing <a class="reference internal" href="#think"><span class="std std-ref">The overwritten think method</span></a>, it suffices to focus on an
additional name space, i.e.
-<a href="#id5"><span class="problematic" id="id6">``</span></a>xlink’’ has to be defined (line ref{ln:namespace}). Once the linking of word
-and corpus is set, the entire MyCoRe object has to be updated. This is done by the functionality of the framework (lines ref{ln:updatestart}–ref{ln:updateend}).
-At the end, an instance of emph{QualityControl} is created.</p>
-<p>%QualityControl
-The class emph{QualityControl} is instantiated with a constructor
-depicted in listing ref{src:constructQC}.
-begin{lstlisting}[language=java,caption={Constructor of QualityControl.java},label=src:constructQC,escapechar=|]
-private MCRObject mycoreObject;
-/* Constructor calls method to carry out quality control, i.e. if at least 20</p>
-<blockquote>
-<div><ul class="simple">
-<li>different users agree 100% on the segments of the word under investigation</li>
-</ul>
-<p><a href="#id7"><span class="problematic" id="id8">*</span></a>/</p>
-</div></blockquote>
-<p>public QualityControl(MCRObject mycoreObject) throws Exception
-{</p>
-<blockquote>
-<div><p>this.mycoreObject = mycoreObject;
-if (getEqualObjectNumber() > 20)
-{</p>
-<blockquote>
-<div>addToMorphiloDB();</div></blockquote>
-<p>}</p>
-</div></blockquote>
-<p>}
-end{lstlisting}
-The constructor takes an MyCoRe object, a potential word candidate for the
+<em>xlink</em> has to be defined (line 60). Once the linking of word
+and corpus is set, the entire MyCoRe object has to be updated. This is done by the functionality of the framework (lines 67–69).
+At the end, an instance of <em>QualityControl</em> is created.</p>
+<p>The class <em>QualityControl</em> is instantiated with a constructor
+depicted in listing <a class="reference internal" href="#constructqc"><span class="std std-ref">Constructor of QualityControl.java</span></a>.</p>
+<div class="literal-block-wrapper docutils container" id="id6">
+<span id="constructqc"></span><div class="code-block-caption"><span class="caption-text">Constructor of QualityControl.java</span><a class="headerlink" href="#id6" title="Permalink to this code">¶</a></div>
+<div class="highlight-java notranslate"><div class="highlight"><pre><span></span><span class="kd">private</span> <span class="n">MCRObject</span> <span class="n">mycoreObject</span><span class="o">;</span>
+<span class="cm">/* Constructor calls method to carry out quality control, i.e. if at least 20</span>
+<span class="cm"> * different users agree 100% on the segments of the word under investigation</span>
+<span class="cm"> */</span>
+<span class="kd">public</span> <span class="nf">QualityControl</span><span class="o">(</span><span class="n">MCRObject</span> <span class="n">mycoreObject</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span>
+<span class="o">{</span>
+ <span class="k">this</span><span class="o">.</span><span class="na">mycoreObject</span> <span class="o">=</span> <span class="n">mycoreObject</span><span class="o">;</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">getEqualObjectNumber</span><span class="o">()</span> <span class="o">></span> <span class="mi">20</span><span class="o">)</span>
+ <span class="o">{</span>
+ <span class="n">addToMorphiloDB</span><span class="o">();</span>
+ <span class="o">}</span>
+<span class="o">}</span>
+</pre></div>
+</div>
+</div>
+<p>The constructor takes an MyCoRe object, a potential word candidate for the
master data base, which is assigned to a private class variable because the
object is used though not changed by some other java methods.
-More importantly, there are two more methods: emph{getEqualNumber()} and
-emph{addToMorphiloDB()}. While the former initiates a process of counting and
+More importantly, there are two more methods: <em>getEqualNumber()</em> and
+<em>addToMorphiloDB()</em>. While the former initiates a process of counting and
comparing objects, the latter is concerned with calculating the correct number
-of occurrences from different, but not the same texts, and generating a MyCoRe object with the same content but with two different flags in the emph{//service/servflags/servflag}-node, i.e. emph{createdby=’administrator’} and emph{state=’published’}.
-And of course, the emph{occurrence} attribute is set to the newly calculated value. The logic corresponds exactly to what was explained in
-listing ref{src:think} and will not be repeated here. The only difference are the paths compiled by the XPathFactory. They are
-begin{itemize}</p>
-<blockquote>
-<div>item[-] emph{//service/servflags/servflag[@type=’createdby’]} and
-item[-] emph{//service/servstates/servstate[@classid=’state’]}.</div></blockquote>
-<p>end{itemize}
-It is more instructive to document how the number of occurrences is calculated. There are two steps involved. First, a list with all mycore objects that are
-equal to the object which the class is instantiated with (<a href="#id9"><span class="problematic" id="id10">``</span></a>mycoreObject’’ in listing ref{src:constructQC}) is created. This list is looped and all occurrence
-attributes are summed up. Second, all occurrences from equal texts are substracted. Equal texts are identified on the basis of its meta data and its derivate.
-There are some obvious shortcomings of this approach, which will be discussed in chapter ref{chap:results}, section ref{sec:improv}. Here, suffice it to
-understand the mode of operation. Listing ref{src:equalOcc} shows a possible solution.
-begin{lstlisting}[language=java,caption={Occurrence Extraction from Equal Texts (1)},label=src:equalOcc,escapechar=|]
-/* returns number of Occurrences if Objects are equal, zero otherwise</p>
-<blockquote>
-<div><a href="#id11"><span class="problematic" id="id12">*</span></a>/</div></blockquote>
-<p>private int getOccurrencesFromEqualTexts(MCRObject mcrobj1, MCRObject mcrobj2) throws SAXException, IOException
-{</p>
-<blockquote>
-<div><p>int occurrences = 1;
-//extract corpmeta ObjectIDs from morphilo-Objects
-String crpID1 = getAttributeValue(“//corpuslink”, “href”, mcrobj1);
-String crpID2 = getAttributeValue(“//corpuslink”, “href”, mcrobj2);
-//get these two corpmeta Objects
-MCRObject corpo1 = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(crpID1));
-MCRObject corpo2 = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(crpID2));
-//are the texts equal? get list of ‘processed-words’ derivate
-String corp1DerivID = getAttributeValue(“//structure/derobjects/derobject”, “href”, corpo1);
-String corp2DerivID = getAttributeValue(“//structure/derobjects/derobject”, “href”, corpo2);</p>
-<p>ArrayList result = new ArrayList(getContentFromFile(corp1DerivID, “”));|label{ln:writeContent}|
-result.remove(getContentFromFile(corp2DerivID, “”));|label{ln:removeContent}|
-if (result.size() == 0) // the texts are equal
-{</p>
-<blockquote>
-<div>// extract occurrences of one the objects
-occurrences = Integer.parseInt(getAttributeValue(“//morphiloContainer/morphilo/w”, “occurrence”, mcrobj1));</div></blockquote>
-<p>}
-else
-{</p>
-<blockquote>
-<div>occurrences = 0; //project metadata happened to be the same, but texts are different</div></blockquote>
-<p>}
-return occurrences;</p>
-</div></blockquote>
-<p>}
-end{lstlisting}
-In this implementation, the ids from the emph{corpmeta} data model are accessed via the xlink attribute in the morphilo documents.
-The method emph{getAttributeValue(String, String, MCRObject)} does exactly the same as demonstrated earlier (see from line ref{ln:namespace}
-on in listing ref{src:getUnknowWords}). The underlying logic is that the texts are equal if exactly the same number of words were uploaded.
-So all words from one file are written to a list (line ref{ln:writeContent}) and words from the other file are removed from the
-very same list (line ref{ln:removeContent}). If this list is empty, then the exact same number of words must have been in both files and the occurrences
+of occurrences from different, but not the same texts, and generating a MyCoRe object with the same
+content but with two different flags in the <em>//service/servflags/servflag</em>-node, i.e. <em>createdby=’administrator’</em> and <em>state=’published’</em>.
+And of course, the <em>occurrence</em> attribute is set to the newly calculated value. The logic corresponds exactly to what was explained in
+listing <a class="reference internal" href="#think"><span class="std std-ref">The overwritten think method</span></a> and will not be repeated here. The only difference are the paths compiled by the XPathFactory. They are</p>
+<ul class="simple">
+<li><em>//service/servflags/servflag[@type=’createdby’]</em> and</li>
+<li><em>//service/servstates/servstate[@classid=’state’]</em>.</li>
+</ul>
+<p>It is more instructive to document how the number of occurrences is calculated. There are two steps involved. First, a list with all mycore objects that are
+equal to the object which the class is instantiated with (<em>mycoreObject</em> in listing <a class="reference internal" href="#constructqc"><span class="std std-ref">Constructor of QualityControl.java</span></a>) is created. This list is looped and all occurrence
+attributes are summed up. Second, all occurrences from equal texts are substracted. Equal texts are identified on the basis of its meta data and its derivate.</p>
+<div class="literal-block-wrapper docutils container" id="id7">
+<div class="code-block-caption"><span class="caption-text">Occurrence Extraction from Equal Texts</span><a class="headerlink" href="#id7" title="Permalink to this code">¶</a></div>
+<div class="highlight-java notranslate"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+10
+11
+12
+13
+14
+15
+16
+17
+18
+19
+20
+21
+22
+23
+24
+25
+26
+27
+28</pre></div></td><td class="code"><div class="highlight"><pre><span></span><span class="cm">/* returns number of Occurrences if Objects are equal, zero otherwise</span>
+<span class="cm"> */</span>
+<span class="kd">private</span> <span class="kt">int</span> <span class="nf">getOccurrencesFromEqualTexts</span><span class="o">(</span><span class="n">MCRObject</span> <span class="n">mcrobj1</span><span class="o">,</span> <span class="n">MCRObject</span> <span class="n">mcrobj2</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">SAXException</span><span class="o">,</span> <span class="n">IOException</span>
+<span class="o">{</span>
+ <span class="kt">int</span> <span class="n">occurrences</span> <span class="o">=</span> <span class="mi">1</span><span class="o">;</span>
+ <span class="c1">//extract corpmeta ObjectIDs from morphilo-Objects</span>
+ <span class="n">String</span> <span class="n">crpID1</span> <span class="o">=</span> <span class="n">getAttributeValue</span><span class="o">(</span><span class="s">"//corpuslink"</span><span class="o">,</span> <span class="s">"href"</span><span class="o">,</span> <span class="n">mcrobj1</span><span class="o">);</span>
+ <span class="n">String</span> <span class="n">crpID2</span> <span class="o">=</span> <span class="n">getAttributeValue</span><span class="o">(</span><span class="s">"//corpuslink"</span><span class="o">,</span> <span class="s">"href"</span><span class="o">,</span> <span class="n">mcrobj2</span><span class="o">);</span>
+ <span class="c1">//get these two corpmeta Objects</span>
+ <span class="n">MCRObject</span> <span class="n">corpo1</span> <span class="o">=</span> <span class="n">MCRMetadataManager</span><span class="o">.</span><span class="na">retrieveMCRObject</span><span class="o">(</span><span class="n">MCRObjectID</span><span class="o">.</span><span class="na">getInstance</span><span class="o">(</span><span class="n">crpID1</span><span class="o">));</span>
+ <span class="n">MCRObject</span> <span class="n">corpo2</span> <span class="o">=</span> <span class="n">MCRMetadataManager</span><span class="o">.</span><span class="na">retrieveMCRObject</span><span class="o">(</span><span class="n">MCRObjectID</span><span class="o">.</span><span class="na">getInstance</span><span class="o">(</span><span class="n">crpID2</span><span class="o">));</span>
+ <span class="c1">//are the texts equal? get list of 'processed-words' derivate</span>
+ <span class="n">String</span> <span class="n">corp1DerivID</span> <span class="o">=</span> <span class="n">getAttributeValue</span><span class="o">(</span><span class="s">"//structure/derobjects/derobject"</span><span class="o">,</span> <span class="s">"href"</span><span class="o">,</span> <span class="n">corpo1</span><span class="o">);</span>
+ <span class="n">String</span> <span class="n">corp2DerivID</span> <span class="o">=</span> <span class="n">getAttributeValue</span><span class="o">(</span><span class="s">"//structure/derobjects/derobject"</span><span class="o">,</span> <span class="s">"href"</span><span class="o">,</span> <span class="n">corpo2</span><span class="o">);</span>
+
+ <span class="n">ArrayList</span> <span class="n">result</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ArrayList</span><span class="o">(</span><span class="n">getContentFromFile</span><span class="o">(</span><span class="n">corp1DerivID</span><span class="o">,</span> <span class="s">""</span><span class="o">));</span>
+ <span class="n">result</span><span class="o">.</span><span class="na">remove</span><span class="o">(</span><span class="n">getContentFromFile</span><span class="o">(</span><span class="n">corp2DerivID</span><span class="o">,</span> <span class="s">""</span><span class="o">));</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">result</span><span class="o">.</span><span class="na">size</span><span class="o">()</span> <span class="o">==</span> <span class="mi">0</span><span class="o">)</span> <span class="c1">// the texts are equal</span>
+ <span class="o">{</span>
+ <span class="c1">// extract occurrences of one the objects</span>
+ <span class="n">occurrences</span> <span class="o">=</span> <span class="n">Integer</span><span class="o">.</span><span class="na">parseInt</span><span class="o">(</span><span class="n">getAttributeValue</span><span class="o">(</span><span class="s">"//morphiloContainer/morphilo/w"</span><span class="o">,</span> <span class="s">"occurrence"</span><span class="o">,</span> <span class="n">mcrobj1</span><span class="o">));</span>
+ <span class="o">}</span>
+ <span class="k">else</span>
+ <span class="o">{</span>
+ <span class="n">occurrences</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span> <span class="c1">//project metadata happened to be the same, but texts are different</span>
+ <span class="o">}</span>
+ <span class="k">return</span> <span class="n">occurrences</span><span class="o">;</span>
+<span class="o">}</span>
+</pre></div>
+</td></tr></table></div>
+</div>
+<p>In this implementation, the ids from the <em>corpmeta</em> data model are accessed via the xlink attribute in the morphilo documents.
+The method <em>getAttributeValue(String, String, MCRObject)</em> does exactly the same as demonstrated earlier (see from line 60
+on in listing <a class="reference internal" href="#getunknowwords"><span class="std std-ref">Mode of Operation of getUnknownWords Method</span></a>). The underlying logic is that the texts are equal if exactly the same number of words were uploaded.
+So all words from one file are written to a list (line 16) and words from the other file are removed from the
+very same list (line 17). If this list is empty, then the exact same number of words must have been in both files and the occurrences
are adjusted accordingly. Since this method is called from another private method that only contains a loop through all equal objects, one gets
-the occurrences from all equal texts. For reasons of confirmability, the looping method is also given:
-begin{lstlisting}[language=java,caption={Occurrence Extraction from Equal Texts (2)},label=src:equalOcc2,escapechar=|]
-private int getOccurrencesFromEqualTexts() throws Exception
-{</p>
-<blockquote>
-<div><p>ArrayList<MCRObject> equalObjects = new ArrayList<MCRObject>();
-equalObjects = getAllEqualMCRObjects();
-int occurrences = 0;
-for (MCRObject obj : equalObjects)
-{</p>
-<blockquote>
-<div>occurrences = occurrences + getOccurrencesFromEqualTexts(mycoreObject, obj);</div></blockquote>
-<p>}
-return occurrences;</p>
-</div></blockquote>
-<p>}
-end{lstlisting}</p>
-<p>Now, the constructor in listing ref{src:constructQC} reveals another method that rolls out an equally complex concatenation of procedures.
-As implied above, emph{getEqualObjectNumber()} returns the number of equally annotated words. It does this by falling back to another
-method from which the size of the returned list is calculated (emph{getAllEqualMCRObjects().size()}). Hence, we should care about
-emph{getAllEqualMCRObjects()}. This method really has the same design as emph{int getOccurrencesFromEqualTexts()} in listing ref{src:equalOcc2}.
-The difference is that another method (emph{Boolean compareMCRObjects(MCRObject, MCRObject, String)}) is used within the loop and
+the occurrences from all equal texts. For reasons of confirmability, the looping method is also given:</p>
+<div class="literal-block-wrapper docutils container" id="id8">
+<span id="equalocc2"></span><div class="code-block-caption"><span class="caption-text">Occurrence Extraction from Equal Texts (2)</span><a class="headerlink" href="#id8" title="Permalink to this code">¶</a></div>
+<div class="highlight-java notranslate"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+10
+11</pre></div></td><td class="code"><div class="highlight"><pre><span></span><span class="kd">private</span> <span class="kt">int</span> <span class="nf">getOccurrencesFromEqualTexts</span><span class="o">()</span> <span class="kd">throws</span> <span class="n">Exception</span>
+<span class="o">{</span>
+ <span class="n">ArrayList</span><span class="o"><</span><span class="n">MCRObject</span><span class="o">></span> <span class="n">equalObjects</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ArrayList</span><span class="o"><</span><span class="n">MCRObject</span><span class="o">>();</span>
+ <span class="n">equalObjects</span> <span class="o">=</span> <span class="n">getAllEqualMCRObjects</span><span class="o">();</span>
+ <span class="kt">int</span> <span class="n">occurrences</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span>
+ <span class="k">for</span> <span class="o">(</span><span class="n">MCRObject</span> <span class="n">obj</span> <span class="o">:</span> <span class="n">equalObjects</span><span class="o">)</span>
+ <span class="o">{</span>
+ <span class="n">occurrences</span> <span class="o">=</span> <span class="n">occurrences</span> <span class="o">+</span> <span class="n">getOccurrencesFromEqualTexts</span><span class="o">(</span><span class="n">mycoreObject</span><span class="o">,</span> <span class="n">obj</span><span class="o">);</span>
+ <span class="o">}</span>
+ <span class="k">return</span> <span class="n">occurrences</span><span class="o">;</span>
+<span class="o">}</span>
+</pre></div>
+</td></tr></table></div>
+</div>
+<p>Now, the constructor in listing <a class="reference internal" href="#constructqc"><span class="std std-ref">Constructor of QualityControl.java</span></a> reveals another method that rolls out an equally complex concatenation of procedures.
+As implied above, <em>getEqualObjectNumber()</em> returns the number of equally annotated words. It does this by falling back to another
+method from which the size of the returned list is calculated (<em>getAllEqualMCRObjects().size()</em>). Hence, we should care about
+<em>getAllEqualMCRObjects()</em>. This method really has the same design as <em>int getOccurrencesFromEqualTexts()</em> in listing <a class="reference internal" href="#equalocc2"><span class="std std-ref">Occurrence Extraction from Equal Texts (2)</span></a>.
+The difference is that another method (<em>Boolean compareMCRObjects(MCRObject, MCRObject, String)</em>) is used within the loop and
that all equal objects are put into the list of MyCoRe objects that are returned. If this list comprises more than 20
-entries,footnote{This number is somewhat arbitrary. It is inspired by the sample size n in t-distributed data.} the respective document
+entries, <a class="footnote-reference" href="#f4" id="id1">[1]</a> the respective document
will be integrated in the master data base by the process described above.
-The comparator logic is shown in listing ref{src:compareMCR}.
-begin{lstlisting}[language=java,caption={Comparison of MyCoRe objects},label=src:compareMCR,escapechar=|]
-private Boolean compareMCRObjects(MCRObject mcrobj1, MCRObject mcrobj2, String xpath) throws SAXException, IOException
-{</p>
-<blockquote>
-<div><p>Boolean isEqual = false;
-Boolean beginTime = false;
-Boolean endTime = false;
-Boolean occDiff = false;
-Boolean corpusDiff = false;</p>
-<p>String source = getXMLFromObject(mcrobj1, xpath);
-String target = getXMLFromObject(mcrobj2, xpath);</p>
-<p>XMLUnit.setIgnoreAttributeOrder(true);
-XMLUnit.setIgnoreComments(true);
-XMLUnit.setIgnoreDiffBetweenTextAndCDATA(true);
-XMLUnit.setIgnoreWhitespace(true);
-XMLUnit.setNormalizeWhitespace(true);</p>
-<p>//differences in occurrences, end, begin should be ignored
-try
-{</p>
-<blockquote>
-<div><p>Diff xmlDiff = new Diff(source, target);
-DetailedDiff dd = new DetailedDiff(xmlDiff);
-//counters for differences
-int i = 0;
-int j = 0;
-int k = 0;
-int l = 0;
-// list containing all differences
-List differences = dd.getAllDifferences();|label{ln:difflist}|
-for (Object object : differences)
-{</p>
-<blockquote>
-<div><p>Difference difference = (Difference) object;
-<a class="reference external" href="mailto://%40begin">//<span>@</span>begin</a>,@end,… node is not in the difference list if the count is 0
-if (difference.getControlNodeDetail().getXpathLocation().endsWith(“@begin”)) i++;|label{ln:diffbegin}|
-if (difference.getControlNodeDetail().getXpathLocation().endsWith(“@end”)) j++;
-if (difference.getControlNodeDetail().getXpathLocation().endsWith(“@occurrence”)) k++;
-if (difference.getControlNodeDetail().getXpathLocation().endsWith(“@corpus”)) l++;|label{ln:diffend}|
-//@begin and @end have different values: they must be checked if they fall right in the allowed time range
-if ( difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())</p>
-<blockquote>
-<div>&& difference.getControlNodeDetail().getXpathLocation().endsWith(“@begin”)
-&& (Integer.parseInt(difference.getControlNodeDetail().getValue()) < Integer.parseInt(difference.getTestNodeDetail().getValue())) )</div></blockquote>
-<dl class="docutils">
-<dt>{</dt>
-<dd>beginTime = true;</dd>
-</dl>
-<p>}
-if (difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())</p>
-<blockquote>
-<div>&& difference.getControlNodeDetail().getXpathLocation().endsWith(“@end”)
-&& (Integer.parseInt(difference.getControlNodeDetail().getValue()) > Integer.parseInt(difference.getTestNodeDetail().getValue())) )</div></blockquote>
-<dl class="docutils">
-<dt>{</dt>
-<dd>endTime = true;</dd>
-</dl>
-<p>}
-//attribute values of @occurrence and @corpus are ignored if they are different
-if (difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())</p>
-<blockquote>
-<div>&& difference.getControlNodeDetail().getXpathLocation().endsWith(“@occurrence”))</div></blockquote>
-<dl class="docutils">
-<dt>{</dt>
-<dd>occDiff = true;</dd>
-</dl>
-<p>}
-if (difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())</p>
-<blockquote>
-<div>&& difference.getControlNodeDetail().getXpathLocation().endsWith(“@corpus”))</div></blockquote>
-<dl class="docutils">
-<dt>{</dt>
-<dd>corpusDiff = true;</dd>
-</dl>
-<p>}</p>
-</div></blockquote>
-<p>}
-//if any of @begin, @end … is identical set Boolean to true
-if (i == 0) beginTime = true;|label{ln:zerobegin}|
-if (j == 0) endTime = true;
-if (k == 0) occDiff = true;
-if (l == 0) corpusDiff = true;|label{ln:zeroend}|
-//if the size of differences is greater than the number of changes admitted in @begin, @end … something else must be different
-if (beginTime && endTime && occDiff && corpusDiff && (i + j + k + l) == dd.getAllDifferences().size()) isEqual = true;|label{ln:diffsum}|
-}
-catch (SAXException e)
-{</p>
-<blockquote>
-<div>e.printStackTrace();</div></blockquote>
-<p>}
-catch (IOException e)
-{</p>
-<blockquote>
-<div>e.printStackTrace();</div></blockquote>
-<p>}</p>
-</div></blockquote>
-<p>return isEqual;</p>
-</div></blockquote>
-<p>}
-end{lstlisting}
-In this method, XMLUnit is heavily used to make all necessary node comparisons. The matter becomes more complicated, however, if some attributes
+The comparator logic is shown in listing <a class="reference internal" href="#comparemcr"><span class="std std-ref">Comparison of MyCoRe objects</span></a>.</p>
+<div class="literal-block-wrapper docutils container" id="id9">
+<span id="comparemcr"></span><div class="code-block-caption"><span class="caption-text">Comparison of MyCoRe objects</span><a class="headerlink" href="#id9" title="Permalink to this code">¶</a></div>
+<div class="highlight-java notranslate"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+10
+11
+12
+13
+14
+15
+16
+17
+18
+19
+20
+21
+22
+23
+24
+25
+26
+27
+28
+29
+30
+31
+32
+33
+34
+35
+36
+37
+38
+39
+40
+41
+42
+43
+44
+45
+46
+47
+48
+49
+50
+51
+52
+53
+54
+55
+56
+57
+58
+59
+60
+61
+62
+63
+64
+65
+66
+67
+68
+69
+70
+71
+72
+73
+74
+75
+76
+77
+78
+79
+80</pre></div></td><td class="code"><div class="highlight"><pre><span></span><span class="kd">private</span> <span class="n">Boolean</span> <span class="nf">compareMCRObjects</span><span class="o">(</span><span class="n">MCRObject</span> <span class="n">mcrobj1</span><span class="o">,</span> <span class="n">MCRObject</span> <span class="n">mcrobj2</span><span class="o">,</span> <span class="n">String</span> <span class="n">xpath</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">SAXException</span><span class="o">,</span> <span class="n">IOException</span>
+<span class="o">{</span>
+ <span class="n">Boolean</span> <span class="n">isEqual</span> <span class="o">=</span> <span class="kc">false</span><span class="o">;</span>
+ <span class="n">Boolean</span> <span class="n">beginTime</span> <span class="o">=</span> <span class="kc">false</span><span class="o">;</span>
+ <span class="n">Boolean</span> <span class="n">endTime</span> <span class="o">=</span> <span class="kc">false</span><span class="o">;</span>
+ <span class="n">Boolean</span> <span class="n">occDiff</span> <span class="o">=</span> <span class="kc">false</span><span class="o">;</span>
+ <span class="n">Boolean</span> <span class="n">corpusDiff</span> <span class="o">=</span> <span class="kc">false</span><span class="o">;</span>
+
+ <span class="n">String</span> <span class="n">source</span> <span class="o">=</span> <span class="n">getXMLFromObject</span><span class="o">(</span><span class="n">mcrobj1</span><span class="o">,</span> <span class="n">xpath</span><span class="o">);</span>
+ <span class="n">String</span> <span class="n">target</span> <span class="o">=</span> <span class="n">getXMLFromObject</span><span class="o">(</span><span class="n">mcrobj2</span><span class="o">,</span> <span class="n">xpath</span><span class="o">);</span>
+
+ <span class="n">XMLUnit</span><span class="o">.</span><span class="na">setIgnoreAttributeOrder</span><span class="o">(</span><span class="kc">true</span><span class="o">);</span>
+ <span class="n">XMLUnit</span><span class="o">.</span><span class="na">setIgnoreComments</span><span class="o">(</span><span class="kc">true</span><span class="o">);</span>
+ <span class="n">XMLUnit</span><span class="o">.</span><span class="na">setIgnoreDiffBetweenTextAndCDATA</span><span class="o">(</span><span class="kc">true</span><span class="o">);</span>
+ <span class="n">XMLUnit</span><span class="o">.</span><span class="na">setIgnoreWhitespace</span><span class="o">(</span><span class="kc">true</span><span class="o">);</span>
+ <span class="n">XMLUnit</span><span class="o">.</span><span class="na">setNormalizeWhitespace</span><span class="o">(</span><span class="kc">true</span><span class="o">);</span>
+
+ <span class="c1">//differences in occurrences, end, begin should be ignored</span>
+ <span class="k">try</span>
+ <span class="o">{</span>
+ <span class="n">Diff</span> <span class="n">xmlDiff</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Diff</span><span class="o">(</span><span class="n">source</span><span class="o">,</span> <span class="n">target</span><span class="o">);</span>
+ <span class="n">DetailedDiff</span> <span class="n">dd</span> <span class="o">=</span> <span class="k">new</span> <span class="n">DetailedDiff</span><span class="o">(</span><span class="n">xmlDiff</span><span class="o">);</span>
+ <span class="c1">//counters for differences</span>
+ <span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span>
+ <span class="kt">int</span> <span class="n">j</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span>
+ <span class="kt">int</span> <span class="n">k</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span>
+ <span class="kt">int</span> <span class="n">l</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span>
+ <span class="c1">// list containing all differences</span>
+<span class="hll"> <span class="n">List</span> <span class="n">differences</span> <span class="o">=</span> <span class="n">dd</span><span class="o">.</span><span class="na">getAllDifferences</span><span class="o">();</span>
+</span> <span class="k">for</span> <span class="o">(</span><span class="n">Object</span> <span class="n">object</span> <span class="o">:</span> <span class="n">differences</span><span class="o">)</span>
+ <span class="o">{</span>
+ <span class="n">Difference</span> <span class="n">difference</span> <span class="o">=</span> <span class="o">(</span><span class="n">Difference</span><span class="o">)</span> <span class="n">object</span><span class="o">;</span>
+ <span class="c1">//@begin,@end,... node is not in the difference list if the count is 0</span>
+<span class="hll"> <span class="k">if</span> <span class="o">(</span><span class="n">difference</span><span class="o">.</span><span class="na">getControlNodeDetail</span><span class="o">().</span><span class="na">getXpathLocation</span><span class="o">().</span><span class="na">endsWith</span><span class="o">(</span><span class="s">"@begin"</span><span class="o">))</span> <span class="n">i</span><span class="o">++;</span>
+</span> <span class="k">if</span> <span class="o">(</span><span class="n">difference</span><span class="o">.</span><span class="na">getControlNodeDetail</span><span class="o">().</span><span class="na">getXpathLocation</span><span class="o">().</span><span class="na">endsWith</span><span class="o">(</span><span class="s">"@end"</span><span class="o">))</span> <span class="n">j</span><span class="o">++;</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">difference</span><span class="o">.</span><span class="na">getControlNodeDetail</span><span class="o">().</span><span class="na">getXpathLocation</span><span class="o">().</span><span class="na">endsWith</span><span class="o">(</span><span class="s">"@occurrence"</span><span class="o">))</span> <span class="n">k</span><span class="o">++;</span>
+<span class="hll"> <span class="k">if</span> <span class="o">(</span><span class="n">difference</span><span class="o">.</span><span class="na">getControlNodeDetail</span><span class="o">().</span><span class="na">getXpathLocation</span><span class="o">().</span><span class="na">endsWith</span><span class="o">(</span><span class="s">"@corpus"</span><span class="o">))</span> <span class="n">l</span><span class="o">++;</span>
+</span> <span class="c1">//@begin and @end have different values: they must be checked if they fall right in the allowed time range</span>
+ <span class="k">if</span> <span class="o">(</span> <span class="n">difference</span><span class="o">.</span><span class="na">getControlNodeDetail</span><span class="o">().</span><span class="na">getXpathLocation</span><span class="o">().</span><span class="na">equals</span><span class="o">(</span><span class="n">difference</span><span class="o">.</span><span class="na">getTestNodeDetail</span><span class="o">().</span><span class="na">getXpathLocation</span><span class="o">())</span>
+ <span class="o">&&</span> <span class="n">difference</span><span class="o">.</span><span class="na">getControlNodeDetail</span><span class="o">().</span><span class="na">getXpathLocation</span><span class="o">().</span><span class="na">endsWith</span><span class="o">(</span><span class="s">"@begin"</span><span class="o">)</span>
+ <span class="o">&&</span> <span class="o">(</span><span class="n">Integer</span><span class="o">.</span><span class="na">parseInt</span><span class="o">(</span><span class="n">difference</span><span class="o">.</span><span class="na">getControlNodeDetail</span><span class="o">().</span><span class="na">getValue</span><span class="o">())</span> <span class="o"><</span> <span class="n">Integer</span><span class="o">.</span><span class="na">parseInt</span><span class="o">(</span><span class="n">difference</span><span class="o">.</span><span class="na">getTestNodeDetail</span><span class="o">().</span><span class="na">getValue</span><span class="o">()))</span> <span class="o">)</span>
+ <span class="o">{</span>
+ <span class="n">beginTime</span> <span class="o">=</span> <span class="kc">true</span><span class="o">;</span>
+ <span class="o">}</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">difference</span><span class="o">.</span><span class="na">getControlNodeDetail</span><span class="o">().</span><span class="na">getXpathLocation</span><span class="o">().</span><span class="na">equals</span><span class="o">(</span><span class="n">difference</span><span class="o">.</span><span class="na">getTestNodeDetail</span><span class="o">().</span><span class="na">getXpathLocation</span><span class="o">())</span>
+ <span class="o">&&</span> <span class="n">difference</span><span class="o">.</span><span class="na">getControlNodeDetail</span><span class="o">().</span><span class="na">getXpathLocation</span><span class="o">().</span><span class="na">endsWith</span><span class="o">(</span><span class="s">"@end"</span><span class="o">)</span>
+ <span class="o">&&</span> <span class="o">(</span><span class="n">Integer</span><span class="o">.</span><span class="na">parseInt</span><span class="o">(</span><span class="n">difference</span><span class="o">.</span><span class="na">getControlNodeDetail</span><span class="o">().</span><span class="na">getValue</span><span class="o">())</span> <span class="o">></span> <span class="n">Integer</span><span class="o">.</span><span class="na">parseInt</span><span class="o">(</span><span class="n">difference</span><span class="o">.</span><span class="na">getTestNodeDetail</span><span class="o">().</span><span class="na">getValue</span><span class="o">()))</span> <span class="o">)</span>
+ <span class="o">{</span>
+ <span class="n">endTime</span> <span class="o">=</span> <span class="kc">true</span><span class="o">;</span>
+ <span class="o">}</span>
+ <span class="c1">//attribute values of @occurrence and @corpus are ignored if they are different</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">difference</span><span class="o">.</span><span class="na">getControlNodeDetail</span><span class="o">().</span><span class="na">getXpathLocation</span><span class="o">().</span><span class="na">equals</span><span class="o">(</span><span class="n">difference</span><span class="o">.</span><span class="na">getTestNodeDetail</span><span class="o">().</span><span class="na">getXpathLocation</span><span class="o">())</span>
+ <span class="o">&&</span> <span class="n">difference</span><span class="o">.</span><span class="na">getControlNodeDetail</span><span class="o">().</span><span class="na">getXpathLocation</span><span class="o">().</span><span class="na">endsWith</span><span class="o">(</span><span class="s">"@occurrence"</span><span class="o">))</span>
+ <span class="o">{</span>
+ <span class="n">occDiff</span> <span class="o">=</span> <span class="kc">true</span><span class="o">;</span>
+ <span class="o">}</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">difference</span><span class="o">.</span><span class="na">getControlNodeDetail</span><span class="o">().</span><span class="na">getXpathLocation</span><span class="o">().</span><span class="na">equals</span><span class="o">(</span><span class="n">difference</span><span class="o">.</span><span class="na">getTestNodeDetail</span><span class="o">().</span><span class="na">getXpathLocation</span><span class="o">())</span>
+ <span class="o">&&</span> <span class="n">difference</span><span class="o">.</span><span class="na">getControlNodeDetail</span><span class="o">().</span><span class="na">getXpathLocation</span><span class="o">().</span><span class="na">endsWith</span><span class="o">(</span><span class="s">"@corpus"</span><span class="o">))</span>
+ <span class="o">{</span>
+ <span class="n">corpusDiff</span> <span class="o">=</span> <span class="kc">true</span><span class="o">;</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+ <span class="c1">//if any of @begin, @end ... is identical set Boolean to true</span>
+<span class="hll"> <span class="k">if</span> <span class="o">(</span><span class="n">i</span> <span class="o">==</span> <span class="mi">0</span><span class="o">)</span> <span class="n">beginTime</span> <span class="o">=</span> <span class="kc">true</span><span class="o">;</span>
+</span> <span class="k">if</span> <span class="o">(</span><span class="n">j</span> <span class="o">==</span> <span class="mi">0</span><span class="o">)</span> <span class="n">endTime</span> <span class="o">=</span> <span class="kc">true</span><span class="o">;</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">k</span> <span class="o">==</span> <span class="mi">0</span><span class="o">)</span> <span class="n">occDiff</span> <span class="o">=</span> <span class="kc">true</span><span class="o">;</span>
+<span class="hll"> <span class="k">if</span> <span class="o">(</span><span class="n">l</span> <span class="o">==</span> <span class="mi">0</span><span class="o">)</span> <span class="n">corpusDiff</span> <span class="o">=</span> <span class="kc">true</span><span class="o">;</span>
+</span> <span class="c1">//if the size of differences is greater than the number of changes admitted in @begin, @end ... something else must be different</span>
+<span class="hll"> <span class="k">if</span> <span class="o">(</span><span class="n">beginTime</span> <span class="o">&&</span> <span class="n">endTime</span> <span class="o">&&</span> <span class="n">occDiff</span> <span class="o">&&</span> <span class="n">corpusDiff</span> <span class="o">&&</span> <span class="o">(</span><span class="n">i</span> <span class="o">+</span> <span class="n">j</span> <span class="o">+</span> <span class="n">k</span> <span class="o">+</span> <span class="n">l</span><span class="o">)</span> <span class="o">==</span> <span class="n">dd</span><span class="o">.</span><span class="na">getAllDifferences</span><span class="o">().</span><span class="na">size</span><span class="o">())</span> <span class="n">isEqual</span> <span class="o">=</span> <span class="kc">true</span><span class="o">;</span>
+</span> <span class="o">}</span>
+ <span class="k">catch</span> <span class="o">(</span><span class="n">SAXException</span> <span class="n">e</span><span class="o">)</span>
+ <span class="o">{</span>
+ <span class="n">e</span><span class="o">.</span><span class="na">printStackTrace</span><span class="o">();</span>
+ <span class="o">}</span>
+ <span class="k">catch</span> <span class="o">(</span><span class="n">IOException</span> <span class="n">e</span><span class="o">)</span>
+ <span class="o">{</span>
+ <span class="n">e</span><span class="o">.</span><span class="na">printStackTrace</span><span class="o">();</span>
+ <span class="o">}</span>
+ <span class="k">return</span> <span class="n">isEqual</span><span class="o">;</span>
+<span class="o">}</span>
+</pre></div>
+</td></tr></table></div>
+</div>
+<p>In this method, XMLUnit is heavily used to make all necessary node comparisons. The matter becomes more complicated, however, if some attributes
are not only ignored, but evaluated according to a given definition as it is the case for the time range. If the evaluator and builder classes are
not to be overwritten entirely because needed for evaluating other nodes of the
xml document, the above solution appears a bit awkward. So there is potential for improvement before the production version is to be programmed.</p>
<p>XMLUnit provides us with a
-list of the differences of the two documents (see line ref{ln:difflist}). There are four differences allowed, that is, the attributes emph{occurrence},
-emph{corpus}, emph{begin}, and emph{end}. For each of them a Boolean variable is set. Because any of the attributes could also be equal to the master
+list of the differences of the two documents (see line 29). There are four differences allowed, that is, the attributes <em>occurrence</em>,
+<em>corpus</em>, <em>begin</em>, and <em>end</em>. For each of them a Boolean variable is set. Because any of the attributes could also be equal to the master
document and the difference list only contains the actual differences, one has to find a way to define both, equal and different, for the attributes.
This could be done by ignoring these nodes. Yet, this would not include testing if the beginning and ending dates fall into the range of the master
-document. Therefore the attributes are counted as lines ref{ln:diffbegin} through ref{ln:diffend} reveal. If any two documents
-differ in some of the four attributes just specified, then the sum of the counters (line ref{ln:diffsum}) should not be greater than the collected differences
+document. Therefore the attributes are counted as lines 34 through 37 reveal. If any two documents
+differ in some of the four attributes just specified, then the sum of the counters (line 69) should not be greater than the collected differences
by XMLUnit. The rest of the if-tests assign truth values to the respective
Booleans. It is probably worth mentioning that if all counters are zero (lines
-ref{ln:zerobegin}-ref{ln:zeroend}) the attributes and values are identical and hence the Boolean has to be set explicitly. Otherwise the test in line ref{ln:diffsum} would fail.</p>
-<p>%TagCorpusServlet
-Once quality control (explained in detail further down) has been passed, it is
-the user’s turn to interact further. By clicking on the option emph{Manual tagging}, the emph{TagCorpusServlet} will be callled. This servlet instantiates
-emph{ProcessCorpusServlet} to get access to the emph{getUnknownWords}-method, which delivers the words still to be
-processed and which overwrites the content of the file starting with emph{untagged}. For the next word in emph{leftovers} a new MyCoRe object is created
-using the JDOM API and added to the file beginning with emph{processed}. In line ref{ln:tagmanu} of listing ref{src:tagservlet}, the previously defined
+64–67) the attributes and values are identical and hence the Boolean has to be set explicitly. Otherwise the test in line 69 would fail.</p>
+<p>Once quality control (explained in detail further down) has been passed, it is
+the user’s turn to interact further. By clicking on the option <em>Manual tagging</em>, the <em>TagCorpusServlet</em> will be callled. This servlet instantiates
+<em>ProcessCorpusServlet</em> to get access to the <em>getUnknownWords</em>-method, which delivers the words still to be
+processed and which overwrites the content of the file starting with <em>untagged</em>. For the next word in <em>leftovers</em> a new MyCoRe object is created
+using the JDOM API and added to the file beginning with <em>processed</em>. In line 16 of listing <a class="reference internal" href="#tagservlet"><span class="std std-ref">Manual Tagging Procedure</span></a>, the previously defined
entry mask is called, with which the proposed word structure could be confirmed or changed. How the word structure is determined will be shown later in
-the text.
-begin{lstlisting}[language=java,caption={Manual Tagging Procedure},label=src:tagservlet,escapechar=|]
-…
-if (!leftovers.isEmpty())
-{</p>
-<blockquote>
-<div><p>ArrayList<String> processed = new ArrayList<String>();
-//processed.add(leftovers.get(0));
-JDOMorphilo jdm = new JDOMorphilo();
-MCRObject obj = jdm.createMorphiloObject(job, leftovers.get(0));|label{ln:jdomobject}|
-//write word to be annotated in process list and save it
-Path filePathProc = pcs.getDerivateFilePath(job, “processed”).getFileName();
-Path proc = root.resolve(filePathProc);
-processed = pcs.getContentFromFile(job, “processed”);
-processed.add(leftovers.get(0));
-Files.write(proc, processed);</p>
-<p>//call entry mask for next word
-tagUrl = prop.getBaseURL() + “content/publish/morphilo.xed?id=” + obj.getId();|label{ln:tagmanu}|</p>
-</div></blockquote>
-<p>}
-else
-{</p>
-<blockquote>
-<div><p>//initiate process to give a complete tagged file of the original corpus
-//if untagged-file is empty, match original file with morphilo
-//creator=administrator OR creator=username and write matches in a new file
-ArrayList<String> complete = new ArrayList<String>();
-ProcessCorpusServlet pcs2 = new ProcessCorpusServlet();
-complete = pcs2.getUnknownWords(</p>
-<blockquote>
-<div>pcs2.getContentFromFile(job, “”), //main corpus file
-pcs2.getCorpusMetadata(job, “def.datefrom”),
-pcs2.getCorpusMetadata(job, “def.dateuntil”),
-“”, //wordtype
-false,
-false,
-true);</div></blockquote>
-<p>Files.delete(p);
-MCRXMLFunctions mdm = new MCRXMLFunctions();
-String mainFile = mdm.getMainDocName(derivID);
-Path newRoot = root.resolve(“tagged-” + mainFile);
-Files.write(newRoot, complete);</p>
-<p>//return to Menu page
-tagUrl = prop.getBaseURL() + “receive/” + corpID;</p>
-</div></blockquote>
-<p>}
-end{lstlisting}
-At the point where no more items are in emph{leftsovers} the emph{getUnknownWords}-method is called whereas the last Boolean parameter
+the text.</p>
+<div class="literal-block-wrapper docutils container" id="id10">
+<span id="tagservlet"></span><div class="code-block-caption"><span class="caption-text">Manual Tagging Procedure</span><a class="headerlink" href="#id10" title="Permalink to this code">¶</a></div>
+<div class="highlight-java notranslate"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+10
+11
+12
+13
+14
+15
+16
+17
+18
+19
+20
+21
+22
+23
+24
+25
+26
+27
+28
+29
+30
+31
+32
+33
+34
+35
+36
+37
+38
+39
+40
+41
+42</pre></div></td><td class="code"><div class="highlight"><pre><span></span><span class="o">...</span>
+<span class="k">if</span> <span class="o">(!</span><span class="n">leftovers</span><span class="o">.</span><span class="na">isEmpty</span><span class="o">())</span>
+<span class="o">{</span>
+ <span class="n">ArrayList</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">processed</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ArrayList</span><span class="o"><</span><span class="n">String</span><span class="o">>();</span>
+ <span class="c1">//processed.add(leftovers.get(0));</span>
+ <span class="n">JDOMorphilo</span> <span class="n">jdm</span> <span class="o">=</span> <span class="k">new</span> <span class="n">JDOMorphilo</span><span class="o">();</span>
+ <span class="n">MCRObject</span> <span class="n">obj</span> <span class="o">=</span> <span class="n">jdm</span><span class="o">.</span><span class="na">createMorphiloObject</span><span class="o">(</span><span class="n">job</span><span class="o">,</span> <span class="n">leftovers</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="mi">0</span><span class="o">));</span>
+ <span class="c1">//write word to be annotated in process list and save it</span>
+ <span class="n">Path</span> <span class="n">filePathProc</span> <span class="o">=</span> <span class="n">pcs</span><span class="o">.</span><span class="na">getDerivateFilePath</span><span class="o">(</span><span class="n">job</span><span class="o">,</span> <span class="s">"processed"</span><span class="o">).</span><span class="na">getFileName</span><span class="o">();</span>
+ <span class="n">Path</span> <span class="n">proc</span> <span class="o">=</span> <span class="n">root</span><span class="o">.</span><span class="na">resolve</span><span class="o">(</span><span class="n">filePathProc</span><span class="o">);</span>
+ <span class="n">processed</span> <span class="o">=</span> <span class="n">pcs</span><span class="o">.</span><span class="na">getContentFromFile</span><span class="o">(</span><span class="n">job</span><span class="o">,</span> <span class="s">"processed"</span><span class="o">);</span>
+ <span class="n">processed</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="n">leftovers</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="mi">0</span><span class="o">));</span>
+ <span class="n">Files</span><span class="o">.</span><span class="na">write</span><span class="o">(</span><span class="n">proc</span><span class="o">,</span> <span class="n">processed</span><span class="o">);</span>
+
+ <span class="c1">//call entry mask for next word</span>
+ <span class="n">tagUrl</span> <span class="o">=</span> <span class="n">prop</span><span class="o">.</span><span class="na">getBaseURL</span><span class="o">()</span> <span class="o">+</span> <span class="s">"content/publish/morphilo.xed?id="</span> <span class="o">+</span> <span class="n">obj</span><span class="o">.</span><span class="na">getId</span><span class="o">();</span>
+<span class="o">}</span>
+<span class="k">else</span>
+<span class="o">{</span>
+ <span class="c1">//initiate process to give a complete tagged file of the original corpus</span>
+ <span class="c1">//if untagged-file is empty, match original file with morphilo</span>
+ <span class="c1">//creator=administrator OR creator=username and write matches in a new file</span>
+ <span class="n">ArrayList</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">complete</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ArrayList</span><span class="o"><</span><span class="n">String</span><span class="o">>();</span>
+ <span class="n">ProcessCorpusServlet</span> <span class="n">pcs2</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ProcessCorpusServlet</span><span class="o">();</span>
+ <span class="n">complete</span> <span class="o">=</span> <span class="n">pcs2</span><span class="o">.</span><span class="na">getUnknownWords</span><span class="o">(</span>
+ <span class="n">pcs2</span><span class="o">.</span><span class="na">getContentFromFile</span><span class="o">(</span><span class="n">job</span><span class="o">,</span> <span class="s">""</span><span class="o">),</span> <span class="c1">//main corpus file</span>
+ <span class="n">pcs2</span><span class="o">.</span><span class="na">getCorpusMetadata</span><span class="o">(</span><span class="n">job</span><span class="o">,</span> <span class="s">"def.datefrom"</span><span class="o">),</span>
+ <span class="n">pcs2</span><span class="o">.</span><span class="na">getCorpusMetadata</span><span class="o">(</span><span class="n">job</span><span class="o">,</span> <span class="s">"def.dateuntil"</span><span class="o">),</span>
+ <span class="s">""</span><span class="o">,</span> <span class="c1">//wordtype</span>
+ <span class="kc">false</span><span class="o">,</span>
+ <span class="kc">false</span><span class="o">,</span>
+ <span class="kc">true</span><span class="o">);</span>
+
+ <span class="n">Files</span><span class="o">.</span><span class="na">delete</span><span class="o">(</span><span class="n">p</span><span class="o">);</span>
+ <span class="n">MCRXMLFunctions</span> <span class="n">mdm</span> <span class="o">=</span> <span class="k">new</span> <span class="n">MCRXMLFunctions</span><span class="o">();</span>
+ <span class="n">String</span> <span class="n">mainFile</span> <span class="o">=</span> <span class="n">mdm</span><span class="o">.</span><span class="na">getMainDocName</span><span class="o">(</span><span class="n">derivID</span><span class="o">);</span>
+ <span class="n">Path</span> <span class="n">newRoot</span> <span class="o">=</span> <span class="n">root</span><span class="o">.</span><span class="na">resolve</span><span class="o">(</span><span class="s">"tagged-"</span> <span class="o">+</span> <span class="n">mainFile</span><span class="o">);</span>
+ <span class="n">Files</span><span class="o">.</span><span class="na">write</span><span class="o">(</span><span class="n">newRoot</span><span class="o">,</span> <span class="n">complete</span><span class="o">);</span>
+
+ <span class="c1">//return to Menu page</span>
+ <span class="n">tagUrl</span> <span class="o">=</span> <span class="n">prop</span><span class="o">.</span><span class="na">getBaseURL</span><span class="o">()</span> <span class="o">+</span> <span class="s">"receive/"</span> <span class="o">+</span> <span class="n">corpID</span><span class="o">;</span>
+<span class="o">}</span>
+</pre></div>
+</td></tr></table></div>
+</div>
+<p>At the point where no more items are in <em>leftsovers</em> the <em>getUnknownWords</em>-method is called whereas the last Boolean parameter
is set true. This indicates that the array list containing all available and relevant data to the respective user is returned as seen in
-the code snippet in listing ref{src:writeAll}.
-begin{lstlisting}[language=java,caption={Code snippet to deliver all data to the user},label=src:writeAll,escapechar=|]
-…
-// all data is written to lo in TEI
-if (writeAllData && isAuthorized && timeCorrect)
-{</p>
-<blockquote>
-<div><p>XPathExpression<Element> xpath = xpfac.compile(“//morphiloContainer/morphilo”, Filters.element());
-for (Element e : xpath.evaluate(jdomDoc))
-{</p>
-<blockquote>
-<div>XMLOutputter outputter = new XMLOutputter();
-outputter.setFormat(Format.getPrettyFormat());
-lo.add(outputter.outputString(e.getContent()));</div></blockquote>
-<p>}</p>
-</div></blockquote>
-<div class="section" id="id13">
-<h3>}<a class="headerlink" href="#id13" title="Permalink to this headline">¶</a></h3>
-<p>end{lstlisting}
-The complete list (emph{lo}) is written to yet a third file starting with emph{tagged} and finally returned to the main project webpage.</p>
-<p>%JDOMorphilo
-The interesting question is now where does the word structure come from, which is filled in the entry mask as asserted above.
-In listing ref{src:tagservlet} line ref{ln:jdomobject}, one can see that a JDOM object is created and the method
-emph{createMorphiloObject(MCRServletJob, String)} is called. The string parameter is the word that needs to be analyzed.
-Most of the method is a mere application of the JDOM API given the data model in chapter ref{chap:concept} section
-ref{subsec:datamodel} and listing ref{lst:worddatamodel}. That means namespaces, elements and their attributes are defined in the correct
+the code snippet in listing ref{src:writeAll}.</p>
+<div class="literal-block-wrapper docutils container" id="id11">
+<div class="code-block-caption"><span class="caption-text">Code snippet to deliver all data to the user</span><a class="headerlink" href="#id11" title="Permalink to this code">¶</a></div>
+<div class="highlight-java notranslate"><div class="highlight"><pre><span></span><span class="o">...</span>
+<span class="c1">// all data is written to lo in TEI</span>
+<span class="k">if</span> <span class="o">(</span><span class="n">writeAllData</span> <span class="o">&&</span> <span class="n">isAuthorized</span> <span class="o">&&</span> <span class="n">timeCorrect</span><span class="o">)</span>
+<span class="o">{</span>
+ <span class="n">XPathExpression</span><span class="o"><</span><span class="n">Element</span><span class="o">></span> <span class="n">xpath</span> <span class="o">=</span> <span class="n">xpfac</span><span class="o">.</span><span class="na">compile</span><span class="o">(</span><span class="s">"//morphiloContainer/morphilo"</span><span class="o">,</span> <span class="n">Filters</span><span class="o">.</span><span class="na">element</span><span class="o">());</span>
+ <span class="k">for</span> <span class="o">(</span><span class="n">Element</span> <span class="n">e</span> <span class="o">:</span> <span class="n">xpath</span><span class="o">.</span><span class="na">evaluate</span><span class="o">(</span><span class="n">jdomDoc</span><span class="o">))</span>
+ <span class="o">{</span>
+ <span class="n">XMLOutputter</span> <span class="n">outputter</span> <span class="o">=</span> <span class="k">new</span> <span class="n">XMLOutputter</span><span class="o">();</span>
+ <span class="n">outputter</span><span class="o">.</span><span class="na">setFormat</span><span class="o">(</span><span class="n">Format</span><span class="o">.</span><span class="na">getPrettyFormat</span><span class="o">());</span>
+ <span class="n">lo</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="n">outputter</span><span class="o">.</span><span class="na">outputString</span><span class="o">(</span><span class="n">e</span><span class="o">.</span><span class="na">getContent</span><span class="o">()));</span>
+ <span class="o">}</span>
+<span class="o">}</span>
+<span class="o">...</span>
+</pre></div>
+</div>
+</div>
+<p>The complete list (<em>lo</em>) is written to yet a third file starting with <em>tagged</em> and finally returned to the main project webpage.</p>
+<p>The interesting question is now where does the word structure come from, which is filled in the entry mask as asserted above.
+In listing <a class="reference internal" href="#tagservlet"><span class="std std-ref">Manual Tagging Procedure</span></a> line 7, one can see that a JDOM object is created and the method
+<em>createMorphiloObject(MCRServletJob, String)</em> is called. The string parameter is the word that needs to be analyzed.
+Most of the method is a mere application of the JDOM API given the data model in <a class="reference internal" href="datamodel.html#concept"><span class="std std-ref">Conceptualization</span></a> and listing <a class="reference internal" href="datamodel.html#worddatamodel"><span class="std std-ref">Word Data Model</span></a>. That means namespaces, elements and their attributes are defined in the correct
order and hierarchy.</p>
<p>To fill the elements and attributes with text, i.e. prefixes, suffixes, stems, etc., a Hashmap – containing the morpheme as
key and its position as value – are created that are filled with the results from an AffixStripper instantiation. Depending on how many prefixes
or suffixes respectively are put in the hashmap, the same number of xml elements are created. As a final step, a valid MyCoRe id is generated using
the existing MyCoRe functionality, the object is created and returned to the TagCorpusServlet.</p>
-<p>%AffixStripper explanation
-Last, the analyses of the word structure will be considered. It is implemented
-in the emph{AffixStripper.java} file.
+<p>Last, the analyses of the word structure will be considered. It is implemented
+in the <em>AffixStripper.java</em> file.
All lexical affix morphemes and their allomorphs as well as the inflections were extracted from the
-OEDfootnote{Oxford English Dictionary <a class="reference external" href="http://www.oed.com/">http://www.oed.com/</a>} and saved as enumerated lists (see the example in listing ref{src:enumPref}).
+<a class="reference external" href="http://www.oed.com/">Oxford English Dictionary</a> and saved as enumerated lists (see the example in listing <a class="reference internal" href="#enumpref"><span class="std std-ref">Enumeration Example for the Prefix over</span></a>).
The allomorphic items of these lists are mapped successively to the beginning in the case of prefixes
-(see listing ref{src:analyzePref}, line ref{ln:prefLoop}) or to the end of words in the case of suffixes
-(see listing ref{src:analyzeSuf}). Since each
+(see listing <a class="reference internal" href="#analyzepref"><span class="std std-ref">Method to recognize prefixes</span></a>, line 7) or to the end of words in the case of suffixes
+(see listing <a class="reference internal" href="#analyzesuf"><span class="std std-ref">Cut-off mechanism for suffixes</span></a>). Since each
morphemic variant maps to its morpheme right away, it makes sense to use the morpheme and so
implicitly keep the relation to its allomorph.</p>
-<p>begin{lstlisting}[language=java,caption={Enumeration Example for the Prefix “over”},label=src:enumPref,escapechar=|]
-package custom.mycore.addons.morphilo;</p>
-<p>public enum PrefixEnum {
-…</p>
-<blockquote>
-<div>over(“over”), ufer(“over”), ufor(“over”), uferr(“over”), uvver(“over”), obaer(“over”), ober(“over)”), ofaer(“over”),
-ofere(“over”), ofir(“over”), ofor(“over”), ofer(“over”), ouer(“over”),oferr(“over”), offerr(“over”), offr(“over”), aure(“over”),
-war(“over”), euer(“over”), oferre(“over”), oouer(“over”), oger(“over”), ouere(“over”), ouir(“over”), ouire(“over”),
-ouur(“over”), ouver(“over”), ouyr(“over”), ovar(“over”), overe(“over”), ovre(“over”),ovur(“over”), owuere(“over”), owver(“over”),
-houyr(“over”), ouyre(“over”), ovir(“over”), ovyr(“over”), hover(“over”), auver(“over”), awver(“over”), ovver(“over”),
-hauver(“over”), ova(“over”), ove(“over”), obuh(“over”), ovah(“over”), ovuh(“over”), ofowr(“over”), ouuer(“over”), oure(“over”),
-owere(“over”), owr(“over”), owre(“over”), owur(“over”), owyr(“over”), our(“over”), ower(“over”), oher(“over”),
-ooer(“over”), oor(“over”), owwer(“over”), ovr(“over”), owir(“over”), oar(“over”), aur(“over”), oer(“over”), ufara(“over”),
-ufera(“over”), ufere(“over”), uferra(“over”), ufora(“over”), ufore(“over”), ufra(“over”), ufre(“over”), ufyrra(“over”),
-yfera(“over”), yfere(“over”), yferra(“over”), uuera(“over”), ufe(“over”), uferre(“over”), uuer(“over”), uuere(“over”),
-vfere(“over”), vuer(“over”), vuere(“over”), vver(“over”), uvvor(“over”) …</div></blockquote>
-<dl class="docutils">
-<dt>…chap:results</dt>
-<dd><p class="first">private String morpheme;
-//constructor
-PrefixEnum(String morpheme)
-{</p>
-<blockquote>
-<div>this.morpheme = morpheme;</div></blockquote>
-<p>}
-//getter Method</p>
-<p>public String getMorpheme()
-{</p>
-<blockquote>
-<div>return this.morpheme;</div></blockquote>
-<p class="last">}</p>
-</dd>
-</dl>
-<p>}
-end{lstlisting}
-As can be seen in line ref{ln:prefPutMorph} in listing ref{src:analyzePref}, the morpheme is saved to a hash map together with its position, i.e. the size of the
-map plus one at the time being. In line ref{ln:prefCutoff} the emph{analyzePrefix} method is recursively called until no more matches can be made.</p>
-<p>begin{lstlisting}[language=java,caption={Method to recognize prefixes},label=src:analyzePref,escapechar=|]
-private Map<String, Integer> prefixMorpheme = new HashMap<String,Integer>();
-…
-private void analyzePrefix(String restword)
-{</p>
-<blockquote>
-<div><p>if (!restword.isEmpty()) //Abbruchbedingung fuer Rekursion
-{</p>
-<blockquote>
-<div><p>for (PrefixEnum prefEnum : PrefixEnum.values())|label{ln:prefLoop}|
-{</p>
-<blockquote>
-<div><p>String s = prefEnum.toString();
-if (restword.startsWith(s))
-{</p>
-<blockquote>
-<div>prefixMorpheme.put(s, prefixMorpheme.size() + 1);|label{ln:prefPutMorph}|
-//cut off the prefix that is added to the list
-analyzePrefix(restword.substring(s.length()));|label{ln:prefCutoff}|</div></blockquote>
-<p>}
-else
-{</p>
-<blockquote>
-<div>analyzePrefix(“”);</div></blockquote>
-<p>}</p>
-</div></blockquote>
-<p>}</p>
-</div></blockquote>
-<p>}</p>
-</div></blockquote>
-<p>}
-end{lstlisting}</p>
+<div class="literal-block-wrapper docutils container" id="id12">
+<span id="enumpref"></span><div class="code-block-caption"><span class="caption-text">Enumeration Example for the Prefix <em>over</em></span><a class="headerlink" href="#id12" title="Permalink to this code">¶</a></div>
+<div class="highlight-java notranslate"><div class="highlight"><pre><span></span><span class="kn">package</span> <span class="nn">custom.mycore.addons.morphilo</span><span class="o">;</span>
+<span class="kd">public</span> <span class="kd">enum</span> <span class="n">PrefixEnum</span> <span class="o">{</span>
+<span class="o">...</span>
+ <span class="n">over</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ufer</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ufor</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">uferr</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">uvver</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">obaer</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ober</span><span class="o">(</span><span class="s">"over)"</span><span class="o">),</span> <span class="n">ofaer</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span>
+ <span class="n">ofere</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ofir</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ofor</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ofer</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ouer</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span><span class="n">oferr</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">offerr</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">offr</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">aure</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span>
+ <span class="n">war</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">euer</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">oferre</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">oouer</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">oger</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ouere</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ouir</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ouire</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span>
+ <span class="n">ouur</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ouver</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ouyr</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ovar</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">overe</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ovre</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span><span class="n">ovur</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">owuere</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">owver</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span>
+ <span class="n">houyr</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ouyre</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ovir</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ovyr</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">hover</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">auver</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">awver</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ovver</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span>
+ <span class="n">hauver</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ova</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ove</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">obuh</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ovah</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ovuh</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ofowr</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ouuer</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">oure</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span>
+ <span class="n">owere</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">owr</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">owre</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">owur</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">owyr</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">our</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ower</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">oher</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span>
+ <span class="n">ooer</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">oor</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">owwer</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ovr</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">owir</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">oar</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">aur</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">oer</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ufara</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span>
+ <span class="n">ufera</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ufere</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">uferra</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ufora</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ufore</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ufra</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ufre</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ufyrra</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span>
+ <span class="n">yfera</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">yfere</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">yferra</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">uuera</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">ufe</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">uferre</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">uuer</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">uuere</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span>
+ <span class="n">vfere</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">vuer</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">vuere</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">vver</span><span class="o">(</span><span class="s">"over"</span><span class="o">),</span> <span class="n">uvvor</span><span class="o">(</span><span class="s">"over"</span><span class="o">)</span> <span class="o">...</span>
+ <span class="kd">private</span> <span class="n">String</span> <span class="n">morpheme</span><span class="o">;</span>
+ <span class="c1">//constructor</span>
+ <span class="n">PrefixEnum</span><span class="o">(</span><span class="n">String</span> <span class="n">morpheme</span><span class="o">)</span>
+ <span class="o">{</span>
+ <span class="k">this</span><span class="o">.</span><span class="na">morpheme</span> <span class="o">=</span> <span class="n">morpheme</span><span class="o">;</span>
+ <span class="o">}</span>
+ <span class="c1">//getter Method</span>
+
+ <span class="kd">public</span> <span class="n">String</span> <span class="nf">getMorpheme</span><span class="o">()</span>
+ <span class="o">{</span>
+ <span class="k">return</span> <span class="k">this</span><span class="o">.</span><span class="na">morpheme</span><span class="o">;</span>
+ <span class="o">}</span>
+<span class="o">}</span>
+</pre></div>
+</div>
+</div>
+<p>As can be seen in line 12 in listing <a class="reference internal" href="#analyzepref"><span class="std std-ref">Method to recognize prefixes</span></a>, the morpheme is saved to a hash map together with its position, i.e. the size of the
+map plus one at the time being. In line 14 the <em>analyzePrefix</em> method is recursively called until no more matches can be made.</p>
+<div class="literal-block-wrapper docutils container" id="id13">
+<span id="analyzepref"></span><div class="code-block-caption"><span class="caption-text">Method to recognize prefixes</span><a class="headerlink" href="#id13" title="Permalink to this code">¶</a></div>
+<div class="highlight-java notranslate"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+10
+11
+12
+13
+14
+15
+16
+17
+18
+19
+20
+21
+22</pre></div></td><td class="code"><div class="highlight"><pre><span></span><span class="kd">private</span> <span class="n">Map</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">></span> <span class="n">prefixMorpheme</span> <span class="o">=</span> <span class="k">new</span> <span class="n">HashMap</span><span class="o"><</span><span class="n">String</span><span class="o">,</span><span class="n">Integer</span><span class="o">>();</span>
+<span class="o">...</span>
+<span class="kd">private</span> <span class="kt">void</span> <span class="nf">analyzePrefix</span><span class="o">(</span><span class="n">String</span> <span class="n">restword</span><span class="o">)</span>
+<span class="o">{</span>
+ <span class="k">if</span> <span class="o">(!</span><span class="n">restword</span><span class="o">.</span><span class="na">isEmpty</span><span class="o">())</span> <span class="c1">//Abbruchbedingung fuer Rekursion</span>
+ <span class="o">{</span>
+ <span class="k">for</span> <span class="o">(</span><span class="n">PrefixEnum</span> <span class="n">prefEnum</span> <span class="o">:</span> <span class="n">PrefixEnum</span><span class="o">.</span><span class="na">values</span><span class="o">())</span>
+ <span class="o">{</span>
+ <span class="n">String</span> <span class="n">s</span> <span class="o">=</span> <span class="n">prefEnum</span><span class="o">.</span><span class="na">toString</span><span class="o">();</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">restword</span><span class="o">.</span><span class="na">startsWith</span><span class="o">(</span><span class="n">s</span><span class="o">))</span>
+ <span class="o">{</span>
+<span class="hll"> <span class="n">prefixMorpheme</span><span class="o">.</span><span class="na">put</span><span class="o">(</span><span class="n">s</span><span class="o">,</span> <span class="n">prefixMorpheme</span><span class="o">.</span><span class="na">size</span><span class="o">()</span> <span class="o">+</span> <span class="mi">1</span><span class="o">);</span>
+</span> <span class="c1">//cut off the prefix that is added to the list</span>
+ <span class="n">analyzePrefix</span><span class="o">(</span><span class="n">restword</span><span class="o">.</span><span class="na">substring</span><span class="o">(</span><span class="n">s</span><span class="o">.</span><span class="na">length</span><span class="o">()));</span>
+ <span class="o">}</span>
+ <span class="k">else</span>
+ <span class="o">{</span>
+ <span class="n">analyzePrefix</span><span class="o">(</span><span class="s">""</span><span class="o">);</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+<span class="o">}</span>
+</pre></div>
+</td></tr></table></div>
+</div>
<p>The recognition of suffixes differs only in the cut-off direction since suffixes occur at the end of a word.
-Hence, line ref{ln:prefCutoff} in listing ref{src:analyzePref} reads in the case of suffixes.</p>
-<p>begin{lstlisting}[language=java,caption={Cut-off mechanism for suffixes},label=src:analyzeSuf,escapechar=|]
-analyzeSuffix(restword.substring(0, restword.length() - s.length()));
-end{lstlisting}</p>
+Hence, line 14 in listing <a class="reference internal" href="#analyzepref"><span class="std std-ref">Method to recognize prefixes</span></a> reads in the case of suffixes.</p>
+<div class="literal-block-wrapper docutils container" id="id14">
+<span id="analyzesuf"></span><div class="code-block-caption"><span class="caption-text">Cut-off mechanism for suffixes</span><a class="headerlink" href="#id14" title="Permalink to this code">¶</a></div>
+<div class="highlight-java notranslate"><div class="highlight"><pre><span></span><span class="n">analyzeSuffix</span><span class="o">(</span><span class="n">restword</span><span class="o">.</span><span class="na">substring</span><span class="o">(</span><span class="mi">0</span><span class="o">,</span> <span class="n">restword</span><span class="o">.</span><span class="na">length</span><span class="o">()</span> <span class="o">-</span> <span class="n">s</span><span class="o">.</span><span class="na">length</span><span class="o">()));</span>
+</pre></div>
+</div>
+</div>
<p>It is important to note that inflections are suffixes (in the given model case of Middle English morphology) that usually occur at the very
end of a word, i.e. after all lexical suffixes, only once. It follows that inflections
have to be recognized at first without any repetition. So the procedure for inflections can be simplified
-to a substantial degree as listing ref{src:analyzeInfl} shows.</p>
-<p>begin{lstlisting}[language=java,caption={Method to recognize inflections},label=src:analyzeInfl,escapechar=|]
-private String analyzeInflection(String wrd)
-{</p>
-<blockquote>
-<div><p>String infl = “”;
-for (InflectionEnum inflEnum : InflectionEnum.values())
-{</p>
-<blockquote>
-<div><p>if (wrd.endsWith(inflEnum.toString()))
-{</p>
-<blockquote>
-<div>infl = inflEnum.toString();</div></blockquote>
-<p>}</p>
-</div></blockquote>
-<p>}
-return infl;</p>
-</div></blockquote>
-<p>}
-end{lstlisting}</p>
+to a substantial degree as listing <a class="reference internal" href="#analyzeinfl"><span class="std std-ref">Method to recognize inflections</span></a> shows.</p>
+<div class="literal-block-wrapper docutils container" id="id15">
+<span id="analyzeinfl"></span><div class="code-block-caption"><span class="caption-text">Method to recognize inflections</span><a class="headerlink" href="#id15" title="Permalink to this code">¶</a></div>
+<div class="highlight-java notranslate"><div class="highlight"><pre><span></span><span class="kd">private</span> <span class="n">String</span> <span class="nf">analyzeInflection</span><span class="o">(</span><span class="n">String</span> <span class="n">wrd</span><span class="o">)</span>
+<span class="o">{</span>
+ <span class="n">String</span> <span class="n">infl</span> <span class="o">=</span> <span class="s">""</span><span class="o">;</span>
+ <span class="k">for</span> <span class="o">(</span><span class="n">InflectionEnum</span> <span class="n">inflEnum</span> <span class="o">:</span> <span class="n">InflectionEnum</span><span class="o">.</span><span class="na">values</span><span class="o">())</span>
+ <span class="o">{</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">wrd</span><span class="o">.</span><span class="na">endsWith</span><span class="o">(</span><span class="n">inflEnum</span><span class="o">.</span><span class="na">toString</span><span class="o">()))</span>
+ <span class="o">{</span>
+ <span class="n">infl</span> <span class="o">=</span> <span class="n">inflEnum</span><span class="o">.</span><span class="na">toString</span><span class="o">();</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+ <span class="k">return</span> <span class="n">infl</span><span class="o">;</span>
+<span class="o">}</span>
+</pre></div>
+</div>
+</div>
<p>Unfortunately the embeddedness problem prevents a very simple algorithm. Embeddedness occurs when a lexical item
-is a substring of another lexical item. To illustrate, the suffix emph{ion} is also contained in the suffix emph{ation}, as is
-emph{ent} in emph{ment}, and so on. The embeddedness problem cannot be solved completely on the basis of linear modelling, but
+is a substring of another lexical item. To illustrate, the suffix <em>ion</em> is also contained in the suffix <em>ation</em>, as is
+<em>ent</em> in <em>ment</em>, and so on. The embeddedness problem cannot be solved completely on the basis of linear modelling, but
for a large part of embedded items one can work around it using implicitly Zipf’s law, i.e. the correlation between frequency
and length of lexical items. The longer a word becomes, the less frequent it will occur. The simplest logic out of it is to assume
that longer suffixes (measured in letters) are preferred over shorter suffixes because it is more likely tha the longer the suffix string becomes,
-the more likely it is one (as opposed to several) suffix unit(s). This is done in listing ref{src:embedAffix}, whereas
-the inner class emph{sortedByLengthMap} returns a list sorted by length and the loop from line ref{ln:deleteAffix} onwards deletes
+the more likely it is one (as opposed to several) suffix unit(s). This is done in listing <a class="reference internal" href="#embedaffix"><span class="std std-ref">Method to workaround embeddedness</span></a>, whereas
+the inner class <em>sortedByLengthMap</em> returns a list sorted by length and the loop from line 17 onwards deletes
the respective substrings.</p>
-<p>begin{lstlisting}[language=java,caption={Method to workaround embeddedness},label=src:embedAffix,escapechar=|]
-private Map<String, Integer> sortOutAffixes(Map<String, Integer> affix)
-{</p>
-<blockquote>
-<div><dl class="docutils">
-<dt>Map<String,Integer> sortedByLengthMap = new TreeMap<String, Integer>(new Comparator<String>()</dt>
-<dd><dl class="first docutils">
-<dt>{</dt>
-<dd><p class="first">@Override
-public int compare(String s1, String s2)
-{</p>
-<blockquote>
-<div>int cmp = Integer.compare(s1.length(), s2.length());
-return cmp != 0 ? cmp : s1.compareTo(s2);</div></blockquote>
-<p class="last">}</p>
-</dd>
-</dl>
-<p class="last">}</p>
-</dd>
-</dl>
-<p>);
-sortedByLengthMap.putAll(affix);
-ArrayList<String> al1 = new ArrayList<String>(sortedByLengthMap.keySet());
-ArrayList<String> al2 = al1;
-Collections.reverse(al2);
-for (String s2 : al1)|label{ln:deleteAffix}|
-{</p>
-<blockquote>
-<div><dl class="docutils">
-<dt>for (String s1 <span class="classifier-delimiter">:</span> <span class="classifier">al2)</span></dt>
-<dd><p class="first">if (s1.contains(s2) && s1.length() > s2.length())
-{</p>
-<blockquote>
-<div>affix.remove(s2);</div></blockquote>
-<p class="last">}</p>
-</dd>
-</dl>
-<p>}</p>
-</div></blockquote>
-<p>return affix;</p>
-</div></blockquote>
-<p>}
-end{lstlisting}</p>
-<p>Finally, the position of the affix has to be calculated because the hashmap in line ref{ln:prefPutMorph} in
-listing ref{src:analyzePref} does not keep the original order for changes taken place in addressing the affix embeddedness
-(listing ref{src:embedAffix}). Listing ref{src:affixPos} depicts the preferred solution.
-The recursive construction of the method is similar to emph{private void analyzePrefix(String)} (listing ref{src:analyzePref})
-only that the two affix types are handled in one method. For that, an additional parameter taking the form either emph{suffix}
-or emph{prefix} is included.</p>
-<p>begin{lstlisting}[language=java,caption={Method to determine position of the affix},label=src:affixPos,escapechar=|]
-private void getAffixPosition(Map<String, Integer> affix, String restword, int pos, String affixtype)
-{</p>
-<blockquote>
-<div><p>if (!restword.isEmpty()) //Abbruchbedingung fuer Rekursion
-{</p>
-<blockquote>
-<div><p>for (String s : affix.keySet())
-{</p>
-<blockquote>
-<div><p>if (restword.startsWith(s) && affixtype.equals(“prefix”))
-{</p>
-<blockquote>
-<div><blockquote>
-<div>pos++;
-prefixMorpheme.put(s, pos);</div></blockquote>
-<dl class="docutils">
-<dt>//prefixAllomorph.add(pos-1, restword.substring(s.length()));</dt>
-<dd>getAffixPosition(affix, restword.substring(s.length()), pos, affixtype);</dd>
-</dl>
-</div></blockquote>
-<p>}
-else if (restword.endsWith(s) && affixtype.equals(“suffix”))
-{</p>
-<blockquote>
-<div>pos++;
-suffixMorpheme.put(s, pos);
-//suffixAllomorph.add(pos-1, restword.substring(s.length()));
-getAffixPosition(affix, restword.substring(0, restword.length() - s.length()), pos, affixtype);</div></blockquote>
-<p>}
-else
-{</p>
-<blockquote>
-<div>getAffixPosition(affix, “”, pos, affixtype);</div></blockquote>
-<p>}</p>
-</div></blockquote>
-<p>}</p>
-</div></blockquote>
-<p>}</p>
-</div></blockquote>
-<p>}
-end{lstlisting}</p>
-<p>To give the complete word structure, the root of a word should also be provided. In listing ref{src:rootAnalyze} a simple solution is offered, however,
-considering compounds as words consisting of more than one root.
-begin{lstlisting}[language=java,caption={Method to determine roots},label=src:rootAnalyze,escapechar=|]
-private ArrayList<String> analyzeRoot(Map<String, Integer> pref, Map<String, Integer> suf, int stemNumber)
-{</p>
-<blockquote>
-<div><p>ArrayList<String> root = new ArrayList<String>();
-int j = 1; //one root always exists
-// if word is a compound several roots exist
-while (j <= stemNumber)
-{</p>
-<blockquote>
-<div><p>j++;
-String rest = lemma;|label{ln:lemma}|</p>
-<p>for (int i=0;i<pref.size();i++)
-{</p>
-<blockquote>
-<div><p>for (String s : pref.keySet())
-{</p>
-<blockquote>
-<div><dl class="docutils">
-<dt>//if (i == pref.get(s))</dt>
-<dd><p class="first">if (rest.length() > s.length() && s.equals(rest.substring(0, s.length())))
-{</p>
-<blockquote class="last">
-<div>rest = rest.substring(s.length(),rest.length());</div></blockquote>
-</dd>
-</dl>
-<p>}</p>
-</div></blockquote>
-<p>}</p>
-</div></blockquote>
-<p>}</p>
-<p>for (int i=0;i<suf.size();i++)
-{</p>
-<blockquote>
-<div><p>for (String s : suf.keySet())
-{</p>
-<blockquote>
-<div><p>//if (i == suf.get(s))
-if (s.length() < rest.length() && (s.equals(rest.substring(rest.length() - s.length(), rest.length()))))
-{</p>
-<blockquote>
-<div>rest = rest.substring(0, rest.length() - s.length());</div></blockquote>
-<p>}</p>
-</div></blockquote>
-<p>}</p>
-</div></blockquote>
-<p>}
-root.add(rest);</p>
-</div></blockquote>
-<p>}
-return root;</p>
-</div></blockquote>
-<p>}
-end{lstlisting}
-The logic behind this method is that the root is the remainder of a word when all prefixes and suffixes are substracted.
+<div class="literal-block-wrapper docutils container" id="id16">
+<span id="embedaffix"></span><div class="code-block-caption"><span class="caption-text">Method to workaround embeddedness</span><a class="headerlink" href="#id16" title="Permalink to this code">¶</a></div>
+<div class="highlight-java notranslate"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+10
+11
+12
+13
+14
+15
+16
+17
+18
+19
+20
+21
+22
+23
+24
+25
+26</pre></div></td><td class="code"><div class="highlight"><pre><span></span><span class="kd">private</span> <span class="n">Map</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">></span> <span class="nf">sortOutAffixes</span><span class="o">(</span><span class="n">Map</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">></span> <span class="n">affix</span><span class="o">)</span>
+<span class="o">{</span>
+ <span class="n">Map</span><span class="o"><</span><span class="n">String</span><span class="o">,</span><span class="n">Integer</span><span class="o">></span> <span class="n">sortedByLengthMap</span> <span class="o">=</span> <span class="k">new</span> <span class="n">TreeMap</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">>(</span><span class="k">new</span> <span class="n">Comparator</span><span class="o"><</span><span class="n">String</span><span class="o">>()</span>
+ <span class="o">{</span>
+ <span class="nd">@Override</span>
+ <span class="kd">public</span> <span class="kt">int</span> <span class="nf">compare</span><span class="o">(</span><span class="n">String</span> <span class="n">s1</span><span class="o">,</span> <span class="n">String</span> <span class="n">s2</span><span class="o">)</span>
+ <span class="o">{</span>
+ <span class="kt">int</span> <span class="n">cmp</span> <span class="o">=</span> <span class="n">Integer</span><span class="o">.</span><span class="na">compare</span><span class="o">(</span><span class="n">s1</span><span class="o">.</span><span class="na">length</span><span class="o">(),</span> <span class="n">s2</span><span class="o">.</span><span class="na">length</span><span class="o">());</span>
+ <span class="k">return</span> <span class="n">cmp</span> <span class="o">!=</span> <span class="mi">0</span> <span class="o">?</span> <span class="n">cmp</span> <span class="o">:</span> <span class="n">s1</span><span class="o">.</span><span class="na">compareTo</span><span class="o">(</span><span class="n">s2</span><span class="o">);</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+ <span class="o">);</span>
+ <span class="n">sortedByLengthMap</span><span class="o">.</span><span class="na">putAll</span><span class="o">(</span><span class="n">affix</span><span class="o">);</span>
+ <span class="n">ArrayList</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">al1</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ArrayList</span><span class="o"><</span><span class="n">String</span><span class="o">>(</span><span class="n">sortedByLengthMap</span><span class="o">.</span><span class="na">keySet</span><span class="o">());</span>
+ <span class="n">ArrayList</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">al2</span> <span class="o">=</span> <span class="n">al1</span><span class="o">;</span>
+ <span class="n">Collections</span><span class="o">.</span><span class="na">reverse</span><span class="o">(</span><span class="n">al2</span><span class="o">);</span>
+ <span class="k">for</span> <span class="o">(</span><span class="n">String</span> <span class="n">s2</span> <span class="o">:</span> <span class="n">al1</span><span class="o">)</span>
+ <span class="o">{</span>
+ <span class="k">for</span> <span class="o">(</span><span class="n">String</span> <span class="n">s1</span> <span class="o">:</span> <span class="n">al2</span><span class="o">)</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">s1</span><span class="o">.</span><span class="na">contains</span><span class="o">(</span><span class="n">s2</span><span class="o">)</span> <span class="o">&&</span> <span class="n">s1</span><span class="o">.</span><span class="na">length</span><span class="o">()</span> <span class="o">></span> <span class="n">s2</span><span class="o">.</span><span class="na">length</span><span class="o">())</span>
+ <span class="o">{</span>
+ <span class="n">affix</span><span class="o">.</span><span class="na">remove</span><span class="o">(</span><span class="n">s2</span><span class="o">);</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+ <span class="k">return</span> <span class="n">affix</span><span class="o">;</span>
+<span class="o">}</span>
+</pre></div>
+</td></tr></table></div>
+</div>
+<p>Finally, the position of the affix has to be calculated because the hashmap in line 12 in
+listing <a class="reference internal" href="#analyzepref"><span class="std std-ref">Method to recognize prefixes</span></a> does not keep the original order for changes taken place in addressing the affix embeddedness
+(listing <a class="reference internal" href="#embedaffix"><span class="std std-ref">Method to workaround embeddedness</span></a>). Listing <a class="reference internal" href="#affixpos"><span class="std std-ref">Method to determine position of the affix</span></a> depicts the preferred solution.
+The recursive construction of the method is similar to <em>private void analyzePrefix(String)</em> (listing <a class="reference internal" href="#analyzepref"><span class="std std-ref">Method to recognize prefixes</span></a>)
+only that the two affix types are handled in one method. For that, an additional parameter taking the form either <em>suffix</em>
+or <em>prefix</em> is included.</p>
+<div class="literal-block-wrapper docutils container" id="id17">
+<span id="affixpos"></span><div class="code-block-caption"><span class="caption-text">Method to determine position of the affix</span><a class="headerlink" href="#id17" title="Permalink to this code">¶</a></div>
+<div class="highlight-java notranslate"><div class="highlight"><pre><span></span><span class="kd">private</span> <span class="kt">void</span> <span class="nf">getAffixPosition</span><span class="o">(</span><span class="n">Map</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">></span> <span class="n">affix</span><span class="o">,</span> <span class="n">String</span> <span class="n">restword</span><span class="o">,</span> <span class="kt">int</span> <span class="n">pos</span><span class="o">,</span> <span class="n">String</span> <span class="n">affixtype</span><span class="o">)</span>
+<span class="o">{</span>
+ <span class="k">if</span> <span class="o">(!</span><span class="n">restword</span><span class="o">.</span><span class="na">isEmpty</span><span class="o">())</span> <span class="c1">//Abbruchbedingung fuer Rekursion</span>
+ <span class="o">{</span>
+ <span class="k">for</span> <span class="o">(</span><span class="n">String</span> <span class="n">s</span> <span class="o">:</span> <span class="n">affix</span><span class="o">.</span><span class="na">keySet</span><span class="o">())</span>
+ <span class="o">{</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">restword</span><span class="o">.</span><span class="na">startsWith</span><span class="o">(</span><span class="n">s</span><span class="o">)</span> <span class="o">&&</span> <span class="n">affixtype</span><span class="o">.</span><span class="na">equals</span><span class="o">(</span><span class="s">"prefix"</span><span class="o">))</span>
+ <span class="o">{</span>
+ <span class="n">pos</span><span class="o">++;</span>
+ <span class="n">prefixMorpheme</span><span class="o">.</span><span class="na">put</span><span class="o">(</span><span class="n">s</span><span class="o">,</span> <span class="n">pos</span><span class="o">);</span>
+ <span class="c1">//prefixAllomorph.add(pos-1, restword.substring(s.length()));</span>
+ <span class="n">getAffixPosition</span><span class="o">(</span><span class="n">affix</span><span class="o">,</span> <span class="n">restword</span><span class="o">.</span><span class="na">substring</span><span class="o">(</span><span class="n">s</span><span class="o">.</span><span class="na">length</span><span class="o">()),</span> <span class="n">pos</span><span class="o">,</span> <span class="n">affixtype</span><span class="o">);</span>
+ <span class="o">}</span>
+ <span class="k">else</span> <span class="k">if</span> <span class="o">(</span><span class="n">restword</span><span class="o">.</span><span class="na">endsWith</span><span class="o">(</span><span class="n">s</span><span class="o">)</span> <span class="o">&&</span> <span class="n">affixtype</span><span class="o">.</span><span class="na">equals</span><span class="o">(</span><span class="s">"suffix"</span><span class="o">))</span>
+ <span class="o">{</span>
+ <span class="n">pos</span><span class="o">++;</span>
+ <span class="n">suffixMorpheme</span><span class="o">.</span><span class="na">put</span><span class="o">(</span><span class="n">s</span><span class="o">,</span> <span class="n">pos</span><span class="o">);</span>
+ <span class="c1">//suffixAllomorph.add(pos-1, restword.substring(s.length()));</span>
+ <span class="n">getAffixPosition</span><span class="o">(</span><span class="n">affix</span><span class="o">,</span> <span class="n">restword</span><span class="o">.</span><span class="na">substring</span><span class="o">(</span><span class="mi">0</span><span class="o">,</span> <span class="n">restword</span><span class="o">.</span><span class="na">length</span><span class="o">()</span> <span class="o">-</span> <span class="n">s</span><span class="o">.</span><span class="na">length</span><span class="o">()),</span> <span class="n">pos</span><span class="o">,</span> <span class="n">affixtype</span><span class="o">);</span>
+ <span class="o">}</span>
+ <span class="k">else</span>
+ <span class="o">{</span>
+ <span class="n">getAffixPosition</span><span class="o">(</span><span class="n">affix</span><span class="o">,</span> <span class="s">""</span><span class="o">,</span> <span class="n">pos</span><span class="o">,</span> <span class="n">affixtype</span><span class="o">);</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+<span class="o">}</span>
+</pre></div>
+</div>
+</div>
+<p>To give the complete word structure, the root of a word should also be provided. In listing <a class="reference internal" href="#rootanalyze"><span class="std std-ref">Method to determine roots</span></a> a simple solution is offered, however,
+considering compounds as words consisting of more than one root.</p>
+<div class="literal-block-wrapper docutils container" id="id18">
+<span id="rootanalyze"></span><div class="code-block-caption"><span class="caption-text">Method to determine roots</span><a class="headerlink" href="#id18" title="Permalink to this code">¶</a></div>
+<div class="highlight-java notranslate"><div class="highlight"><pre><span></span><span class="kd">private</span> <span class="n">ArrayList</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="nf">analyzeRoot</span><span class="o">(</span><span class="n">Map</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">></span> <span class="n">pref</span><span class="o">,</span> <span class="n">Map</span><span class="o"><</span><span class="n">String</span><span class="o">,</span> <span class="n">Integer</span><span class="o">></span> <span class="n">suf</span><span class="o">,</span> <span class="kt">int</span> <span class="n">stemNumber</span><span class="o">)</span>
+<span class="o">{</span>
+ <span class="n">ArrayList</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">root</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ArrayList</span><span class="o"><</span><span class="n">String</span><span class="o">>();</span>
+ <span class="kt">int</span> <span class="n">j</span> <span class="o">=</span> <span class="mi">1</span><span class="o">;</span> <span class="c1">//one root always exists</span>
+ <span class="c1">// if word is a compound several roots exist</span>
+ <span class="k">while</span> <span class="o">(</span><span class="n">j</span> <span class="o"><=</span> <span class="n">stemNumber</span><span class="o">)</span>
+ <span class="o">{</span>
+ <span class="n">j</span><span class="o">++;</span>
+ <span class="n">String</span> <span class="n">rest</span> <span class="o">=</span> <span class="n">lemma</span><span class="o">;</span>
+
+ <span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">i</span><span class="o">=</span><span class="mi">0</span><span class="o">;</span><span class="n">i</span><span class="o"><</span><span class="n">pref</span><span class="o">.</span><span class="na">size</span><span class="o">();</span><span class="n">i</span><span class="o">++)</span>
+ <span class="o">{</span>
+ <span class="k">for</span> <span class="o">(</span><span class="n">String</span> <span class="n">s</span> <span class="o">:</span> <span class="n">pref</span><span class="o">.</span><span class="na">keySet</span><span class="o">())</span>
+ <span class="o">{</span>
+ <span class="c1">//if (i == pref.get(s))</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">rest</span><span class="o">.</span><span class="na">length</span><span class="o">()</span> <span class="o">></span> <span class="n">s</span><span class="o">.</span><span class="na">length</span><span class="o">()</span> <span class="o">&&</span> <span class="n">s</span><span class="o">.</span><span class="na">equals</span><span class="o">(</span><span class="n">rest</span><span class="o">.</span><span class="na">substring</span><span class="o">(</span><span class="mi">0</span><span class="o">,</span> <span class="n">s</span><span class="o">.</span><span class="na">length</span><span class="o">())))</span>
+ <span class="o">{</span>
+ <span class="n">rest</span> <span class="o">=</span> <span class="n">rest</span><span class="o">.</span><span class="na">substring</span><span class="o">(</span><span class="n">s</span><span class="o">.</span><span class="na">length</span><span class="o">(),</span><span class="n">rest</span><span class="o">.</span><span class="na">length</span><span class="o">());</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+
+ <span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">i</span><span class="o">=</span><span class="mi">0</span><span class="o">;</span><span class="n">i</span><span class="o"><</span><span class="n">suf</span><span class="o">.</span><span class="na">size</span><span class="o">();</span><span class="n">i</span><span class="o">++)</span>
+ <span class="o">{</span>
+ <span class="k">for</span> <span class="o">(</span><span class="n">String</span> <span class="n">s</span> <span class="o">:</span> <span class="n">suf</span><span class="o">.</span><span class="na">keySet</span><span class="o">())</span>
+ <span class="o">{</span>
+ <span class="c1">//if (i == suf.get(s))</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">s</span><span class="o">.</span><span class="na">length</span><span class="o">()</span> <span class="o"><</span> <span class="n">rest</span><span class="o">.</span><span class="na">length</span><span class="o">()</span> <span class="o">&&</span> <span class="o">(</span><span class="n">s</span><span class="o">.</span><span class="na">equals</span><span class="o">(</span><span class="n">rest</span><span class="o">.</span><span class="na">substring</span><span class="o">(</span><span class="n">rest</span><span class="o">.</span><span class="na">length</span><span class="o">()</span> <span class="o">-</span> <span class="n">s</span><span class="o">.</span><span class="na">length</span><span class="o">(),</span> <span class="n">rest</span><span class="o">.</span><span class="na">length</span><span class="o">()))))</span>
+ <span class="o">{</span>
+ <span class="n">rest</span> <span class="o">=</span> <span class="n">rest</span><span class="o">.</span><span class="na">substring</span><span class="o">(</span><span class="mi">0</span><span class="o">,</span> <span class="n">rest</span><span class="o">.</span><span class="na">length</span><span class="o">()</span> <span class="o">-</span> <span class="n">s</span><span class="o">.</span><span class="na">length</span><span class="o">());</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+ <span class="n">root</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="n">rest</span><span class="o">);</span>
+ <span class="o">}</span>
+ <span class="k">return</span> <span class="n">root</span><span class="o">;</span>
+<span class="o">}</span>
+</pre></div>
+</div>
+</div>
+<p>The logic behind this method is that the root is the remainder of a word when all prefixes and suffixes are substracted.
So the loops run through the number of prefixes and suffixes at each position and substract the affix. Really, there is
some code doubling with the previously described methods, which could be eliminated by making it more modular in a possible
-refactoring phase. Again, this is not the concern of a prototype. Line ref{ln:lemma} defines the initial state of a root,
-which is the case for monomorphemic words. The emph{lemma} is defined as the wordtoken without the inflection. Thus listing
-ref{src:lemmaAnalyze} reveals how the class variable is calculated
-begin{lstlisting}[language=java,caption={Method to determine lemma},label=src:lemmaAnalyze,escapechar=|]
-/*</p>
-<blockquote>
-<div><ul class="simple">
-<li>Simplification: lemma = wordtoken - inflection</li>
-</ul>
-<p><a href="#id14"><span class="problematic" id="id15">*</span></a>/</p>
-</div></blockquote>
-<p>private String analyzeLemma(String wrd, String infl)
-{</p>
-<blockquote>
-<div>return wrd.substring(0, wrd.length() - infl.length());</div></blockquote>
-<p>}
-end{lstlisting}
-The constructor of emph{AffixStripper} calls the method emph{analyzeWord()}
+refactoring phase. Again, this is not the concern of a prototype. Line 9 defines the initial state of a root,
+which is the case for monomorphemic words. The <em>lemma</em> is defined as the wordtoken without the inflection. Thus listing
+<a class="reference internal" href="#lemmaanalyze"><span class="std std-ref">Method to determine lemma</span></a> reveals how the class variable is calculated</p>
+<div class="literal-block-wrapper docutils container" id="id19">
+<span id="lemmaanalyze"></span><div class="code-block-caption"><span class="caption-text">Method to determine lemma</span><a class="headerlink" href="#id19" title="Permalink to this code">¶</a></div>
+<div class="highlight-java notranslate"><div class="highlight"><pre><span></span><span class="cm">/*</span>
+<span class="cm"> * Simplification: lemma = wordtoken - inflection</span>
+<span class="cm"> */</span>
+<span class="kd">private</span> <span class="n">String</span> <span class="nf">analyzeLemma</span><span class="o">(</span><span class="n">String</span> <span class="n">wrd</span><span class="o">,</span> <span class="n">String</span> <span class="n">infl</span><span class="o">)</span>
+<span class="o">{</span>
+ <span class="k">return</span> <span class="n">wrd</span><span class="o">.</span><span class="na">substring</span><span class="o">(</span><span class="mi">0</span><span class="o">,</span> <span class="n">wrd</span><span class="o">.</span><span class="na">length</span><span class="o">()</span> <span class="o">-</span> <span class="n">infl</span><span class="o">.</span><span class="na">length</span><span class="o">());</span>
+<span class="o">}</span>
+</pre></div>
+</div>
+</div>
+<p>The constructor of <em>AffixStripper</em> calls the method <em>analyzeWord()</em>
whose only job is to calculate each structure element in the correct order
-(listing ref{src:lemmaAnalyze}). All structure elements are also provided by getters.
-begin{lstlisting}[language=java,caption={Method to determine all word structure},label=src:lemmaAnalyze,escapechar=|]
-private void analyzeWord()
-{</p>
-<blockquote>
-<div>//analyze inflection first because it always occurs at the end of a word
-inflection = analyzeInflection(wordtoken);
-lemma = analyzeLemma(wordtoken, inflection);
-analyzePrefix(lemma);
-analyzeSuffix(lemma);
-getAffixPosition(sortOutAffixes(prefixMorpheme), lemma, 0, “prefix”);
-getAffixPosition(sortOutAffixes(suffixMorpheme), lemma, 0, “suffix”);
-prefixNumber = prefixMorpheme.size();
-suffixNumber = suffixMorpheme.size();
-wordroot = analyzeRoot(prefixMorpheme, suffixMorpheme, getStemNumber());</div></blockquote>
-<p>}
-end{lstlisting}</p>
+(listing <a class="reference internal" href="#lemmaanalyze"><span class="std std-ref">Method to determine lemma</span></a>. All structure elements are also provided by getters.</p>
+<div class="literal-block-wrapper docutils container" id="id20">
+<div class="code-block-caption"><span class="caption-text">Method to determine all word structure</span><a class="headerlink" href="#id20" title="Permalink to this code">¶</a></div>
+<div class="highlight-java notranslate"><div class="highlight"><pre><span></span><span class="kd">private</span> <span class="kt">void</span> <span class="nf">analyzeWord</span><span class="o">()</span>
+<span class="o">{</span>
+ <span class="c1">//analyze inflection first because it always occurs at the end of a word</span>
+ <span class="n">inflection</span> <span class="o">=</span> <span class="n">analyzeInflection</span><span class="o">(</span><span class="n">wordtoken</span><span class="o">);</span>
+ <span class="n">lemma</span> <span class="o">=</span> <span class="n">analyzeLemma</span><span class="o">(</span><span class="n">wordtoken</span><span class="o">,</span> <span class="n">inflection</span><span class="o">);</span>
+ <span class="n">analyzePrefix</span><span class="o">(</span><span class="n">lemma</span><span class="o">);</span>
+ <span class="n">analyzeSuffix</span><span class="o">(</span><span class="n">lemma</span><span class="o">);</span>
+ <span class="n">getAffixPosition</span><span class="o">(</span><span class="n">sortOutAffixes</span><span class="o">(</span><span class="n">prefixMorpheme</span><span class="o">),</span> <span class="n">lemma</span><span class="o">,</span> <span class="mi">0</span><span class="o">,</span> <span class="s">"prefix"</span><span class="o">);</span>
+ <span class="n">getAffixPosition</span><span class="o">(</span><span class="n">sortOutAffixes</span><span class="o">(</span><span class="n">suffixMorpheme</span><span class="o">),</span> <span class="n">lemma</span><span class="o">,</span> <span class="mi">0</span><span class="o">,</span> <span class="s">"suffix"</span><span class="o">);</span>
+ <span class="n">prefixNumber</span> <span class="o">=</span> <span class="n">prefixMorpheme</span><span class="o">.</span><span class="na">size</span><span class="o">();</span>
+ <span class="n">suffixNumber</span> <span class="o">=</span> <span class="n">suffixMorpheme</span><span class="o">.</span><span class="na">size</span><span class="o">();</span>
+ <span class="n">wordroot</span> <span class="o">=</span> <span class="n">analyzeRoot</span><span class="o">(</span><span class="n">prefixMorpheme</span><span class="o">,</span> <span class="n">suffixMorpheme</span><span class="o">,</span> <span class="n">getStemNumber</span><span class="o">());</span>
+<span class="o">}</span>
+</pre></div>
+</div>
+</div>
<p>To conclude, the Morphilo implementation as presented here, aims at fulfilling the task of a working prototype. It is important to note
that it neither claims to be a very efficient nor a ready software program to be used in production. However, it marks a crucial milestone
on the way to a production system. At some listings sources of improvement were made explicit; at others no suggestions were made. In the latter
case this does not imply that there is no potential for improvement. Once acceptability tests are carried out, it will be the task of a follow up project
to identify these potentials and implement them accordingly.</p>
-</div>
+<p class="rubric">Notes</p>
+<table class="docutils footnote" frame="void" id="f4" rules="none">
+<colgroup><col class="label" /><col /></colgroup>
+<tbody valign="top">
+<tr><td class="label"><a class="fn-backref" href="#id1">[1]</a></td><td>This number is somewhat arbitrary. It is inspired by the sample size n in t-distributed data.</td></tr>
+</tbody>
+</table>
</div>
</div>
@@ -1012,10 +1254,7 @@ to identify these potentials and implement them accordingly.</p>
<li><a class="reference internal" href="#">Controller Adjustments</a><ul>
<li><a class="reference internal" href="#general-principle-of-operation">General Principle of Operation</a></li>
<li><a class="reference internal" href="#conceptualization">Conceptualization</a></li>
-<li><a class="reference internal" href="#implementation">Implementation</a><ul>
-<li><a class="reference internal" href="#id13">}</a></li>
-</ul>
-</li>
+<li><a class="reference internal" href="#implementation">Implementation</a></li>
</ul>
</li>
</ul>
@@ -1023,8 +1262,7 @@ to identify these potentials and implement them accordingly.</p>
<h3>Related Topics</h3>
<ul>
<li><a href="../index.html">Documentation overview</a><ul>
- <li>Previous: <a href="datamodel.html" title="previous chapter">Data Model</a></li>
- <li>Next: <a href="view.html" title="next chapter">View</a></li>
+ <li>Previous: <a href="view.html" title="previous chapter">View</a></li>
</ul></li>
</ul>
</div>
diff --git a/Morphilo_doc/_build/html/source/datamodel.html b/Morphilo_doc/_build/html/source/datamodel.html
index ed8ca884744fbcaca1943ccd4ae64c3683dd30ea..6c9e17d3434bb3373eca289db2a732e46888e9a6 100644
--- a/Morphilo_doc/_build/html/source/datamodel.html
+++ b/Morphilo_doc/_build/html/source/datamodel.html
@@ -15,8 +15,8 @@
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="Controller Adjustments" href="controller.html" />
- <link rel="prev" title="Documentation Morphilo Project" href="../index.html" />
+ <link rel="next" title="View" href="view.html" />
+ <link rel="prev" title="Software Design" href="architecture.html" />
<link rel="stylesheet" href="../_static/custom.css" type="text/css" />
@@ -34,34 +34,30 @@
<div class="section" id="data-model">
<h1>Data Model<a class="headerlink" href="#data-model" title="Permalink to this headline">¶</a></h1>
<div class="section" id="conceptualization">
-<h2>Conceptualization<a class="headerlink" href="#conceptualization" title="Permalink to this headline">¶</a></h2>
+<span id="concept"></span><h2>Conceptualization<a class="headerlink" href="#conceptualization" title="Permalink to this headline">¶</a></h2>
<p>From both the user and task requirements one can derive that four basic
functions of data processing need to be carried out. Data have to be read, persistently
saved, searched, and deleted. Furthermore, some kind of user management
and multi-user processing is necessary. In addition, the framework should
support web technologies, be well documented, and easy to extent. Ideally, the
MVC pattern is realized.</p>
-<p>subsection{Data Model}label{subsec:datamodel}
-The guidelines of the
-emph{TEI}-standardfootnote{http://www.tei-c.org/release/doc/tei-p5-doc/en/Guidelines.pdf} on the
-word level are defined in line with the structure defined above in section ref{subsec:morphologicalSystems}.
-In listing ref{lst:teiExamp} an
+<p>The guidelines of the <a class="reference external" href="http://www.tei-c.org/release/doc/tei-p5-doc/en/Guidelines.pdf">TEI standard</a> on the
+word level are defined in line with the defined word structure.
+In listing <a class="reference internal" href="#teiexamp"><span class="std std-ref">TEI-example for comfortable</span></a> an
example is given for a possible markup at the word level for
-emph{comfortable}.footnote{http://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-m.html}</p>
-<p>begin{lstlisting}[language=XML,
-caption={TEI-example for ‘comfortable’},label=lst:teiExamp]
-<w type=”adjective”></p>
-<blockquote>
-<div><dl class="docutils">
-<dt><m type=”base”></dt>
-<dd><m type=”prefix” baseForm=”con”>com</m>
-<m type=”root”>fort</m></dd>
-</dl>
-<p></m>
-<m type=”suffix”>able</m></p>
-</div></blockquote>
-<p></w>
-end{lstlisting}</p>
+<a class="reference external" href="http://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-m.html">comfortable</a></p>
+<div class="literal-block-wrapper docutils container" id="id1">
+<span id="teiexamp"></span><div class="code-block-caption"><span class="caption-text">TEI-example for <em>comfortable</em></span><a class="headerlink" href="#id1" title="Permalink to this code">¶</a></div>
+<div class="highlight-xml notranslate"><div class="highlight"><pre><span></span><span class="nt"><w</span> <span class="na">type=</span><span class="s">"adjective"</span><span class="nt">></span>
+ <span class="nt"><m</span> <span class="na">type=</span><span class="s">"base"</span><span class="nt">></span>
+ <span class="nt"><m</span> <span class="na">type=</span><span class="s">"prefix"</span> <span class="na">baseForm=</span><span class="s">"con"</span><span class="nt">></span>com<span class="nt"></m></span>
+ <span class="nt"><m</span> <span class="na">type=</span><span class="s">"root"</span><span class="nt">></span>fort<span class="nt"></m></span>
+ <span class="nt"></m></span>
+ <span class="nt"><m</span> <span class="na">type=</span><span class="s">"suffix"</span><span class="nt">></span>able<span class="nt"></m></span>
+<span class="nt"></w></span>
+</pre></div>
+</div>
+</div>
<p>This data model reflects just one theoretical conception of a word structure model.
Crucially, the model emanates from the assumption
that the suffix node is on par with the word base. On the one hand, this
@@ -70,7 +66,7 @@ other hand, is enclosed in the base, which basically means a stronger lexical,
and less abstract, attachment to the root of a word. Modeling prefixes and suffixes on different
hierarchical levels has important consequences for the branching direction at
subword level (here right-branching). Left the theoretical interest aside, the
-choice of the TEI standard is reasonable with view to a sustainable architecture that allows for
+choice of the <em>TEI</em>-standard is reasonable with view to a sustainable architecture that allows for
exchanging data with little to no additional adjustments.</p>
<p>The negative account is that the model is not eligible for all languages.
It reflects a theoretical construction based on Indo-European
@@ -81,254 +77,295 @@ stem and corresponds to the overwhelming majority of all research carried out
</div>
<div class="section" id="implementation">
<h2>Implementation<a class="headerlink" href="#implementation" title="Permalink to this headline">¶</a></h2>
-<p>As laid out in the task analysis in section ref{subsec:datamodel}, it is
-advantageous to use established standards. It was also shown that it makes sense
+<p>It is
+advantageous to use established standardsn and it makes sense
to keep the meta data of each corpus separate from the data model used for the
words to be analyzed.</p>
-<p>For the present case, the TEI-standard was identified as an
+<p>For the present case, the <em>TEI</em>-standard was identified as an
appropriate markup for words. In terms of the implementation this means that
-the TEI guidelines have to be implemented as an object type compatible with the chosen
+the <em>TEI</em>-guidelines have to be implemented as an object type compatible with the chosen
repository framework. However, the TEI standard is not complete regarding the
diachronic dimension, i.e. information on the development of the word. To
be compatible with the elements of the TEI standard on the one hand
and to best meet the requirements of the application on the other hand, some attributes
are added. This solution allows for processing the xml files according to
-the TEI standard by ignoring the additional attributes and at the same
+the <em>TEI</em>-standard by ignoring the additional attributes and at the same
time, if needed, additional markup can be extracted. The additional attributes
-comprise a link to the corpus meta data, but also emph{position} and
-emph{occurrence} of the affixes.
+comprise a link to the corpus meta data, but also <em>position</em> and
+<em>occurrence</em> of the affixes.
Information on the position and some quantification thereof are potentially relevant for a
wealth of research questions, such as predictions on the productivity of
derivatives and their interaction with the phonological or syntactic modules. So they were included
with respect to future use.</p>
<p>For reasons of efficiency in subsequent processing,
-the historic dates emph{begin} and emph{end} were included in both the word
+the historic dates <em>begin</em> and <em>end</em> were included in both the word
data model and the corpus data model. The result of the word data model is given
-in listing ref{lst:worddatamodel}.
+in listing <a class="reference internal" href="#worddatamodel"><span class="std std-ref">Word Data Model</span></a>.
Whereas attributes of the objecttype are specific to the repository framework, the TEI structure can be
recognized in the hierarchy of the meta data element starting with the name
-emph{w} (line ref{src:wordbegin}).</p>
-<p>begin{lstlisting}[language=XML,caption={Word Data
-model},label=lst:worddatamodel,escapechar=|] <?xml version=”1.0” encoding=”UTF-8”?>
-<objecttype</p>
-<blockquote>
-<div><p>name=”morphilo”
-isChild=”true”
-isParent=”true”
-hasDerivates=”true”
-xmlns:xs=”http://www.w3.org/2001/XMLSchema”
-xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance”
-xsi:noNamespaceSchemaLocation=”datamodel.xsd”>
-<metadata></p>
-<blockquote>
-<div><element name=”morphiloContainer” type=”xml” style=”dontknow”</div></blockquote>
-<dl class="docutils">
-<dt>notinherit=”true” heritable=”false”></dt>
-<dd><blockquote class="first">
-<div><dl class="docutils">
-<dt><xs:sequence></dt>
-<dd><dl class="first docutils">
-<dt><xs:element name=”morphilo”></dt>
-<dd><dl class="first docutils">
-<dt><xs:complexType></dt>
-<dd><dl class="first docutils">
-<dt><xs:sequence></dt>
-<dd><dl class="first docutils">
-<dt><xs:element name=”w” minOccurs=”0” maxOccurs=”unbounded”>|label{src:wordbegin}|</dt>
-<dd><dl class="first docutils">
-<dt><xs:complexType mixed=”true”></dt>
-<dd><dl class="first docutils">
-<dt><xs:sequence></dt>
-<dd><p class="first"><!– stem –>
-<xs:element name=”m1” minOccurs=”0” maxOccurs=”unbounded”></p>
-<blockquote>
-<div><dl class="docutils">
-<dt><xs:complexType mixed=”true”></dt>
-<dd><dl class="first docutils">
-<dt><xs:sequence></dt>
-<dd><p class="first"><!– base –>
-<xs:element name=”m2” minOccurs=”0” maxOccurs=”unbounded”></p>
-<blockquote>
-<div><dl class="docutils">
-<dt><xs:complexType mixed=”true”></dt>
-<dd><dl class="first docutils">
-<dt><xs:sequence></dt>
-<dd><p class="first"><!– root –>
-<xs:element name=”m3” minOccurs=”0” maxOccurs=”unbounded”></p>
-<blockquote>
-<div><dl class="docutils">
-<dt><xs:complexType mixed=”true”></dt>
-<dd><xs:attribute name=”type” type=”xs:string”/></dd>
-</dl>
-<p></xs:complexType></p>
-</div></blockquote>
-<p></xs:element>
-<!– prefix –>
-<xs:element name=”m4” minOccurs=”0” maxOccurs=”unbounded”></p>
-<blockquote>
-<div><dl class="docutils">
-<dt><xs:complexType mixed=”true”></dt>
-<dd><xs:attribute name=”type” type=”xs:string”/>
-<xs:attribute name=”PrefixbaseForm” type=”xs:string”/>
-<xs:attribute name=”position” type=”xs:string”/></dd>
-</dl>
-<p></xs:complexType></p>
-</div></blockquote>
-<p class="last"></xs:element></p>
-</dd>
-</dl>
-<p class="last"></xs:sequence>
-<xs:attribute name=”type” type=”xs:string”/></p>
-</dd>
-</dl>
-<p></xs:complexType></p>
-</div></blockquote>
-<p></xs:element>
-<!– suffix –>
-<xs:element name=”m5” minOccurs=”0” maxOccurs=”unbounded”></p>
-<blockquote>
-<div><dl class="docutils">
-<dt><xs:complexType mixed=”true”></dt>
-<dd><xs:attribute name=”type” type=”xs:string”/>
-<xs:attribute name=”SuffixbaseForm” type=”xs:string”/>
-<xs:attribute name=”position” type=”xs:string”/>
-<xs:attribute name=”inflection” type=”xs:string”/></dd>
-</dl>
-<p></xs:complexType></p>
-</div></blockquote>
-<p class="last"></xs:element></p>
-</dd>
-</dl>
-<p class="last"></xs:sequence>
-<!– stem-Attribute –>
-<xs:attribute name=”type” type=”xs:string”/>
-<xs:attribute name=”pos” type=”xs:string”/>
-<xs:attribute name=”occurrence” type=”xs:string”/></p>
-</dd>
-</dl>
-<p></xs:complexType></p>
-</div></blockquote>
-<p class="last"></xs:element></p>
-</dd>
-</dl>
-<p class="last"></xs:sequence>
-<!– w -Attribute auf Wortebene –>
-<xs:attribute name=”lemma” type=”xs:string”/>
-<xs:attribute name=”complexType” type=”xs:string”/>
-<xs:attribute name=”wordtype” type=”xs:string”/>
-<xs:attribute name=”occurrence” type=”xs:string”/>
-<xs:attribute name=”corpus” type=”xs:string”/>
-<xs:attribute name=”begin” type=”xs:string”/>
-<xs:attribute name=”end” type=”xs:string”/></p>
-</dd>
-</dl>
-<p class="last"></xs:complexType></p>
-</dd>
-</dl>
-<p class="last"></xs:element></p>
-</dd>
-</dl>
-<p class="last"></xs:sequence></p>
-</dd>
-</dl>
-<p class="last"></xs:complexType></p>
-</dd>
-</dl>
-<p class="last"></xs:element></p>
-</dd>
-</dl>
-<p></xs:sequence></p>
-</div></blockquote>
-<p></element>
-<element name=”wordtype” type=”classification” minOccurs=”0” maxOccurs=”1”></p>
-<blockquote>
-<div><classification id=”wordtype”/></div></blockquote>
-<p></element>
-<element name=”complexType” type=”classification” minOccurs=”0” maxOccurs=”1”></p>
-<blockquote>
-<div><classification id=”complexType”/></div></blockquote>
-<p></element>
-<element name=”corpus” type=”classification” minOccurs=”0” maxOccurs=”1”></p>
-<blockquote>
-<div><classification id=”corpus”/></div></blockquote>
-<p></element>
-<element name=”pos” type=”classification” minOccurs=”0” maxOccurs=”1”></p>
-<blockquote>
-<div><classification id=”pos”/></div></blockquote>
-<p></element>
-<element name=”PrefixbaseForm” type=”classification” minOccurs=”0”
-maxOccurs=”1”></p>
-<blockquote>
-<div><classification id=”PrefixbaseForm”/></div></blockquote>
-<p></element>
-<element name=”SuffixbaseForm” type=”classification” minOccurs=”0”
-maxOccurs=”1”></p>
-<blockquote>
-<div><classification id=”SuffixbaseForm”/></div></blockquote>
-<p></element>
-<element name=”inflection” type=”classification” minOccurs=”0” maxOccurs=”1”></p>
-<blockquote>
-<div><classification id=”inflection”/></div></blockquote>
-<p></element>
-<element name=”corpuslink” type=”link” minOccurs=”0” maxOccurs=”unbounded” ></p>
-<blockquote>
-<div><target type=”corpmeta”/></div></blockquote>
-<p class="last"></element></p>
-</dd>
-</dl>
-<p></metadata></p>
-</div></blockquote>
-<p></objecttype>
-end{lstlisting}</p>
+<em>w</em> (line 17).</p>
+<div class="literal-block-wrapper docutils container" id="id2">
+<span id="worddatamodel"></span><div class="code-block-caption"><span class="caption-text">Word Data Model</span><a class="headerlink" href="#id2" title="Permalink to this code">¶</a></div>
+<div class="highlight-xml notranslate"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+ 10
+ 11
+ 12
+ 13
+ 14
+ 15
+ 16
+ 17
+ 18
+ 19
+ 20
+ 21
+ 22
+ 23
+ 24
+ 25
+ 26
+ 27
+ 28
+ 29
+ 30
+ 31
+ 32
+ 33
+ 34
+ 35
+ 36
+ 37
+ 38
+ 39
+ 40
+ 41
+ 42
+ 43
+ 44
+ 45
+ 46
+ 47
+ 48
+ 49
+ 50
+ 51
+ 52
+ 53
+ 54
+ 55
+ 56
+ 57
+ 58
+ 59
+ 60
+ 61
+ 62
+ 63
+ 64
+ 65
+ 66
+ 67
+ 68
+ 69
+ 70
+ 71
+ 72
+ 73
+ 74
+ 75
+ 76
+ 77
+ 78
+ 79
+ 80
+ 81
+ 82
+ 83
+ 84
+ 85
+ 86
+ 87
+ 88
+ 89
+ 90
+ 91
+ 92
+ 93
+ 94
+ 95
+ 96
+ 97
+ 98
+ 99
+100
+101
+102
+103
+104
+105</pre></div></td><td class="code"><div class="highlight"><pre><span></span><span class="cp"><?xml version="1.0" encoding="UTF-8"?></span>
+<span class="nt"><objecttype</span>
+ <span class="na">name=</span><span class="s">"morphilo"</span>
+ <span class="na">isChild=</span><span class="s">"true"</span>
+ <span class="na">isParent=</span><span class="s">"true"</span>
+ <span class="na">hasDerivates=</span><span class="s">"true"</span>
+ <span class="na">xmlns:xs=</span><span class="s">"http://www.w3.org/2001/XMLSchema"</span>
+ <span class="na">xmlns:xsi=</span><span class="s">"http://www.w3.org/2001/XMLSchema-instance"</span>
+ <span class="na">xsi:noNamespaceSchemaLocation=</span><span class="s">"datamodel.xsd"</span><span class="nt">></span>
+ <span class="nt"><metadata></span>
+ <span class="nt"><element</span> <span class="na">name=</span><span class="s">"morphiloContainer"</span> <span class="na">type=</span><span class="s">"xml"</span> <span class="na">style=</span><span class="s">"dontknow"</span>
+ <span class="na">notinherit=</span><span class="s">"true"</span> <span class="na">heritable=</span><span class="s">"false"</span><span class="nt">></span>
+ <span class="nt"><xs:sequence></span>
+ <span class="nt"><xs:element</span> <span class="na">name=</span><span class="s">"morphilo"</span><span class="nt">></span>
+ <span class="nt"><xs:complexType></span>
+ <span class="nt"><xs:sequence></span>
+<span class="hll"> <span class="nt"><xs:element</span> <span class="na">name=</span><span class="s">"w"</span> <span class="na">minOccurs=</span><span class="s">"0"</span> <span class="na">maxOccurs=</span><span class="s">"unbounded"</span><span class="nt">></span>
+</span> <span class="nt"><xs:complexType</span> <span class="na">mixed=</span><span class="s">"true"</span><span class="nt">></span>
+ <span class="nt"><xs:sequence></span>
+ <span class="c"><!-- stem --></span>
+ <span class="nt"><xs:element</span> <span class="na">name=</span><span class="s">"m1"</span> <span class="na">minOccurs=</span><span class="s">"0"</span> <span class="na">maxOccurs=</span><span class="s">"unbounded"</span><span class="nt">></span>
+ <span class="nt"><xs:complexType</span> <span class="na">mixed=</span><span class="s">"true"</span><span class="nt">></span>
+ <span class="nt"><xs:sequence></span>
+ <span class="c"><!-- base --></span>
+ <span class="nt"><xs:element</span> <span class="na">name=</span><span class="s">"m2"</span> <span class="na">minOccurs=</span><span class="s">"0"</span> <span class="na">maxOccurs=</span><span class="s">"unbounded"</span><span class="nt">></span>
+ <span class="nt"><xs:complexType</span> <span class="na">mixed=</span><span class="s">"true"</span><span class="nt">></span>
+ <span class="nt"><xs:sequence></span>
+ <span class="c"><!-- root --></span>
+ <span class="nt"><xs:element</span> <span class="na">name=</span><span class="s">"m3"</span> <span class="na">minOccurs=</span><span class="s">"0"</span> <span class="na">maxOccurs=</span><span class="s">"unbounded"</span><span class="nt">></span>
+ <span class="nt"><xs:complexType</span> <span class="na">mixed=</span><span class="s">"true"</span><span class="nt">></span>
+ <span class="nt"><xs:attribute</span> <span class="na">name=</span><span class="s">"type"</span> <span class="na">type=</span><span class="s">"xs:string"</span><span class="nt">/></span>
+ <span class="nt"></xs:complexType></span>
+ <span class="nt"></xs:element></span>
+ <span class="c"><!-- prefix --></span>
+ <span class="nt"><xs:element</span> <span class="na">name=</span><span class="s">"m4"</span> <span class="na">minOccurs=</span><span class="s">"0"</span> <span class="na">maxOccurs=</span><span class="s">"unbounded"</span><span class="nt">></span>
+ <span class="nt"><xs:complexType</span> <span class="na">mixed=</span><span class="s">"true"</span><span class="nt">></span>
+ <span class="nt"><xs:attribute</span> <span class="na">name=</span><span class="s">"type"</span> <span class="na">type=</span><span class="s">"xs:string"</span><span class="nt">/></span>
+ <span class="nt"><xs:attribute</span> <span class="na">name=</span><span class="s">"PrefixbaseForm"</span> <span class="na">type=</span><span class="s">"xs:string"</span><span class="nt">/></span>
+ <span class="nt"><xs:attribute</span> <span class="na">name=</span><span class="s">"position"</span> <span class="na">type=</span><span class="s">"xs:string"</span><span class="nt">/></span>
+ <span class="nt"></xs:complexType></span>
+ <span class="nt"></xs:element></span>
+ <span class="nt"></xs:sequence></span>
+ <span class="nt"><xs:attribute</span> <span class="na">name=</span><span class="s">"type"</span> <span class="na">type=</span><span class="s">"xs:string"</span><span class="nt">/></span>
+ <span class="nt"></xs:complexType></span>
+ <span class="nt"></xs:element></span>
+ <span class="c"><!-- suffix --></span>
+ <span class="nt"><xs:element</span> <span class="na">name=</span><span class="s">"m5"</span> <span class="na">minOccurs=</span><span class="s">"0"</span> <span class="na">maxOccurs=</span><span class="s">"unbounded"</span><span class="nt">></span>
+ <span class="nt"><xs:complexType</span> <span class="na">mixed=</span><span class="s">"true"</span><span class="nt">></span>
+ <span class="nt"><xs:attribute</span> <span class="na">name=</span><span class="s">"type"</span> <span class="na">type=</span><span class="s">"xs:string"</span><span class="nt">/></span>
+ <span class="nt"><xs:attribute</span> <span class="na">name=</span><span class="s">"SuffixbaseForm"</span> <span class="na">type=</span><span class="s">"xs:string"</span><span class="nt">/></span>
+ <span class="nt"><xs:attribute</span> <span class="na">name=</span><span class="s">"position"</span> <span class="na">type=</span><span class="s">"xs:string"</span><span class="nt">/></span>
+ <span class="nt"><xs:attribute</span> <span class="na">name=</span><span class="s">"inflection"</span> <span class="na">type=</span><span class="s">"xs:string"</span><span class="nt">/></span>
+ <span class="nt"></xs:complexType></span>
+ <span class="nt"></xs:element></span>
+ <span class="nt"></xs:sequence></span>
+ <span class="c"><!-- stem-Attribute --></span>
+ <span class="nt"><xs:attribute</span> <span class="na">name=</span><span class="s">"type"</span> <span class="na">type=</span><span class="s">"xs:string"</span><span class="nt">/></span>
+ <span class="nt"><xs:attribute</span> <span class="na">name=</span><span class="s">"pos"</span> <span class="na">type=</span><span class="s">"xs:string"</span><span class="nt">/></span>
+ <span class="nt"><xs:attribute</span> <span class="na">name=</span><span class="s">"occurrence"</span> <span class="na">type=</span><span class="s">"xs:string"</span><span class="nt">/></span>
+ <span class="nt"></xs:complexType></span>
+ <span class="nt"></xs:element></span>
+ <span class="nt"></xs:sequence></span>
+ <span class="c"><!-- w -Attribute auf Wortebene --></span>
+ <span class="nt"><xs:attribute</span> <span class="na">name=</span><span class="s">"lemma"</span> <span class="na">type=</span><span class="s">"xs:string"</span><span class="nt">/></span>
+ <span class="nt"><xs:attribute</span> <span class="na">name=</span><span class="s">"complexType"</span> <span class="na">type=</span><span class="s">"xs:string"</span><span class="nt">/></span>
+ <span class="nt"><xs:attribute</span> <span class="na">name=</span><span class="s">"wordtype"</span> <span class="na">type=</span><span class="s">"xs:string"</span><span class="nt">/></span>
+ <span class="nt"><xs:attribute</span> <span class="na">name=</span><span class="s">"occurrence"</span> <span class="na">type=</span><span class="s">"xs:string"</span><span class="nt">/></span>
+ <span class="nt"><xs:attribute</span> <span class="na">name=</span><span class="s">"corpus"</span> <span class="na">type=</span><span class="s">"xs:string"</span><span class="nt">/></span>
+ <span class="nt"><xs:attribute</span> <span class="na">name=</span><span class="s">"begin"</span> <span class="na">type=</span><span class="s">"xs:string"</span><span class="nt">/></span>
+ <span class="nt"><xs:attribute</span> <span class="na">name=</span><span class="s">"end"</span> <span class="na">type=</span><span class="s">"xs:string"</span><span class="nt">/></span>
+ <span class="nt"></xs:complexType></span>
+ <span class="nt"></xs:element></span>
+ <span class="nt"></xs:sequence></span>
+ <span class="nt"></xs:complexType></span>
+ <span class="nt"></xs:element></span>
+ <span class="nt"></xs:sequence></span>
+ <span class="nt"></element></span>
+ <span class="nt"><element</span> <span class="na">name=</span><span class="s">"wordtype"</span> <span class="na">type=</span><span class="s">"classification"</span> <span class="na">minOccurs=</span><span class="s">"0"</span> <span class="na">maxOccurs=</span><span class="s">"1"</span><span class="nt">></span>
+ <span class="nt"><classification</span> <span class="na">id=</span><span class="s">"wordtype"</span><span class="nt">/></span>
+ <span class="nt"></element></span>
+ <span class="nt"><element</span> <span class="na">name=</span><span class="s">"complexType"</span> <span class="na">type=</span><span class="s">"classification"</span> <span class="na">minOccurs=</span><span class="s">"0"</span> <span class="na">maxOccurs=</span><span class="s">"1"</span><span class="nt">></span>
+ <span class="nt"><classification</span> <span class="na">id=</span><span class="s">"complexType"</span><span class="nt">/></span>
+ <span class="nt"></element></span>
+ <span class="nt"><element</span> <span class="na">name=</span><span class="s">"corpus"</span> <span class="na">type=</span><span class="s">"classification"</span> <span class="na">minOccurs=</span><span class="s">"0"</span> <span class="na">maxOccurs=</span><span class="s">"1"</span><span class="nt">></span>
+ <span class="nt"><classification</span> <span class="na">id=</span><span class="s">"corpus"</span><span class="nt">/></span>
+ <span class="nt"></element></span>
+ <span class="nt"><element</span> <span class="na">name=</span><span class="s">"pos"</span> <span class="na">type=</span><span class="s">"classification"</span> <span class="na">minOccurs=</span><span class="s">"0"</span> <span class="na">maxOccurs=</span><span class="s">"1"</span><span class="nt">></span>
+ <span class="nt"><classification</span> <span class="na">id=</span><span class="s">"pos"</span><span class="nt">/></span>
+ <span class="nt"></element></span>
+ <span class="nt"><element</span> <span class="na">name=</span><span class="s">"PrefixbaseForm"</span> <span class="na">type=</span><span class="s">"classification"</span> <span class="na">minOccurs=</span><span class="s">"0"</span>
+ <span class="na">maxOccurs=</span><span class="s">"1"</span><span class="nt">></span>
+ <span class="nt"><classification</span> <span class="na">id=</span><span class="s">"PrefixbaseForm"</span><span class="nt">/></span>
+ <span class="nt"></element></span>
+ <span class="nt"><element</span> <span class="na">name=</span><span class="s">"SuffixbaseForm"</span> <span class="na">type=</span><span class="s">"classification"</span> <span class="na">minOccurs=</span><span class="s">"0"</span>
+ <span class="na">maxOccurs=</span><span class="s">"1"</span><span class="nt">></span>
+ <span class="nt"><classification</span> <span class="na">id=</span><span class="s">"SuffixbaseForm"</span><span class="nt">/></span>
+ <span class="nt"></element></span>
+ <span class="nt"><element</span> <span class="na">name=</span><span class="s">"inflection"</span> <span class="na">type=</span><span class="s">"classification"</span> <span class="na">minOccurs=</span><span class="s">"0"</span> <span class="na">maxOccurs=</span><span class="s">"1"</span><span class="nt">></span>
+ <span class="nt"><classification</span> <span class="na">id=</span><span class="s">"inflection"</span><span class="nt">/></span>
+ <span class="nt"></element></span>
+ <span class="nt"><element</span> <span class="na">name=</span><span class="s">"corpuslink"</span> <span class="na">type=</span><span class="s">"link"</span> <span class="na">minOccurs=</span><span class="s">"0"</span> <span class="na">maxOccurs=</span><span class="s">"unbounded"</span> <span class="nt">></span>
+ <span class="nt"><target</span> <span class="na">type=</span><span class="s">"corpmeta"</span><span class="nt">/></span>
+ <span class="nt"></element></span>
+ <span class="nt"></metadata></span>
+<span class="nt"></objecttype></span>
+</pre></div>
+</td></tr></table></div>
+</div>
<p>Additionally, it is worth mentioning that some attributes are modeled as a
-emph{classification}. All these have to be listed
+<em>classification</em>. All these have to be listed
as separate elements in the data model. This has been done for all attributes
that are more or less subject to little or no change. In fact, all known suffix
and prefix morphemes should be known for the language investigated and are
therefore defined as a classification.
-The same is true for the parts of speech named emph{pos} in the morphilo data
+The same is true for the parts of speech named <em>pos</em> in the morphilo data
model above.
Here the PENN-Treebank tagset was used. Last, the different morphemic layers in
-the standard model named emph{m} are changed to $m1$ through $m5$. This is the
+the standard model named <em>m</em> are changed to <em>m1</em> through <em>m5</em>. This is the
only change in the standard that could be problematic if the data is to be
processed elsewhere and the change is not documented more explicitly. Yet, this
change was necessary for the MyCoRe repository throws errors caused by ambiguity
-issues on the different $m$-layers.</p>
+issues on the different <em>m</em>-layers.</p>
<p>The second data model describes only very few properties of the text corpora
-from which the words are extracted. Listing ref{lst:corpusdatamodel} depicts
+from which the words are extracted. Listing <a class="reference internal" href="#corpusdatamodel"><span class="std std-ref">Corpus Data Model</span></a> depicts
only the meta data element. For the sake of simplicity of the prototype, this
data model is kept as simple as possible. The obligatory field is the name of
the corpus. Specific dates of the corpus are classified as optional because in
some cases a text cannot be dated reliably.</p>
-<p>begin{lstlisting}[language=XML,caption={Corpus Data
-Model},label=lst:corpusdatamodel]
-<metadata></p>
-<blockquote>
-<div><p><!– Pflichtfelder –>
-<element name=”korpusname” type=”text” minOccurs=”1” maxOccurs=”1”/>
-<!– Optionale Felder –>
-<element name=”sprache” type=”text” minOccurs=”0” maxOccurs=”1”/>
-<element name=”size” type=”number” minOccurs=”0” maxOccurs=”1”/>
-<element name=”datefrom” type=”text” minOccurs=”0” maxOccurs=”1”/>
-<element name=”dateuntil” type=”text” minOccurs=”0” maxOccurs=”1”/>
-<!– number of words –>
-<element name=”NoW” type=”text” minOccurs=”0” maxOccurs=”1”/>
-<element name=”corpuslink” type=”link” minOccurs=”0” maxOccurs=”unbounded”></p>
-<blockquote>
-<div><target type=”morphilo”/></div></blockquote>
-<p></element></p>
-</div></blockquote>
-<p></metadata>
-end{lstlisting}</p>
+<div class="literal-block-wrapper docutils container" id="id3">
+<span id="corpusdatamodel"></span><div class="code-block-caption"><span class="caption-text">Corpus Data Model</span><a class="headerlink" href="#id3" title="Permalink to this code">¶</a></div>
+<div class="highlight-xml notranslate"><div class="highlight"><pre><span></span><span class="nt"><metadata></span>
+ <span class="c"><!-- Pflichtfelder --></span>
+ <span class="nt"><element</span> <span class="na">name=</span><span class="s">"korpusname"</span> <span class="na">type=</span><span class="s">"text"</span> <span class="na">minOccurs=</span><span class="s">"1"</span> <span class="na">maxOccurs=</span><span class="s">"1"</span><span class="nt">/></span>
+ <span class="c"><!-- Optionale Felder --></span>
+ <span class="nt"><element</span> <span class="na">name=</span><span class="s">"sprache"</span> <span class="na">type=</span><span class="s">"text"</span> <span class="na">minOccurs=</span><span class="s">"0"</span> <span class="na">maxOccurs=</span><span class="s">"1"</span><span class="nt">/></span>
+ <span class="nt"><element</span> <span class="na">name=</span><span class="s">"size"</span> <span class="na">type=</span><span class="s">"number"</span> <span class="na">minOccurs=</span><span class="s">"0"</span> <span class="na">maxOccurs=</span><span class="s">"1"</span><span class="nt">/></span>
+ <span class="nt"><element</span> <span class="na">name=</span><span class="s">"datefrom"</span> <span class="na">type=</span><span class="s">"text"</span> <span class="na">minOccurs=</span><span class="s">"0"</span> <span class="na">maxOccurs=</span><span class="s">"1"</span><span class="nt">/></span>
+ <span class="nt"><element</span> <span class="na">name=</span><span class="s">"dateuntil"</span> <span class="na">type=</span><span class="s">"text"</span> <span class="na">minOccurs=</span><span class="s">"0"</span> <span class="na">maxOccurs=</span><span class="s">"1"</span><span class="nt">/></span>
+ <span class="c"><!-- number of words --></span>
+ <span class="nt"><element</span> <span class="na">name=</span><span class="s">"NoW"</span> <span class="na">type=</span><span class="s">"text"</span> <span class="na">minOccurs=</span><span class="s">"0"</span> <span class="na">maxOccurs=</span><span class="s">"1"</span><span class="nt">/></span>
+ <span class="nt"><element</span> <span class="na">name=</span><span class="s">"corpuslink"</span> <span class="na">type=</span><span class="s">"link"</span> <span class="na">minOccurs=</span><span class="s">"0"</span> <span class="na">maxOccurs=</span><span class="s">"unbounded"</span><span class="nt">></span>
+ <span class="nt"><target</span> <span class="na">type=</span><span class="s">"morphilo"</span><span class="nt">/></span>
+ <span class="nt"></element></span>
+<span class="nt"></metadata></span>
+</pre></div>
+</div>
+</div>
<p>As a final remark, one might have noticed that all attributes are modelled as
strings although other data types are available and fields encoding the dates or
the number of words suggest otherwise. The MyCoRe framework even provides a
-data type emph{historydate}. There is not a very satisfying answer to its
+data type <em>historydate</em>. There is not a very satisfying answer to its
disuse.
All that can be said is that the use of data types different than the string
leads later on to problems in the convergence between the search engine and the
repository framework. These issues seem to be well known and can be followed on
-github.</p>
+<a class="reference external" href="https://github.com/MyCoRe-Org">github</a>.</p>
</div>
</div>
@@ -350,8 +387,8 @@ github.</p>
<h3>Related Topics</h3>
<ul>
<li><a href="../index.html">Documentation overview</a><ul>
- <li>Previous: <a href="../index.html" title="previous chapter">Documentation Morphilo Project</a></li>
- <li>Next: <a href="controller.html" title="next chapter">Controller Adjustments</a></li>
+ <li>Previous: <a href="architecture.html" title="previous chapter">Software Design</a></li>
+ <li>Next: <a href="view.html" title="next chapter">View</a></li>
</ul></li>
</ul>
</div>
diff --git a/Morphilo_doc/_build/html/source/framework.html b/Morphilo_doc/_build/html/source/framework.html
deleted file mode 100644
index d10a21930aa3b32961f54778cc3aa2309e4d4da2..0000000000000000000000000000000000000000
--- a/Morphilo_doc/_build/html/source/framework.html
+++ /dev/null
@@ -1,114 +0,0 @@
-
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
- "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-
-<html xmlns="http://www.w3.org/1999/xhtml">
- <head>
- <meta http-equiv="X-UA-Compatible" content="IE=Edge" />
- <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
- <title>Framework — Morphilo documentation</title>
- <link rel="stylesheet" href="../_static/alabaster.css" type="text/css" />
- <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
- <script type="text/javascript" src="../_static/documentation_options.js"></script>
- <script type="text/javascript" src="../_static/jquery.js"></script>
- <script type="text/javascript" src="../_static/underscore.js"></script>
- <script type="text/javascript" src="../_static/doctools.js"></script>
- <link rel="index" title="Index" href="../genindex.html" />
- <link rel="search" title="Search" href="../search.html" />
- <link rel="prev" title="Software Design" href="architecture.html" />
-
- <link rel="stylesheet" href="../_static/custom.css" type="text/css" />
-
-
- <meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
-
- </head><body>
-
-
- <div class="document">
- <div class="documentwrapper">
- <div class="bodywrapper">
- <div class="body" role="main">
-
- <div class="section" id="framework">
-<h1>Framework<a class="headerlink" href="#framework" title="Permalink to this headline">¶</a></h1>
-<dl class="docutils">
-<dt>begin{figure}</dt>
-<dd>centering
-includegraphics[scale=0.33]{mycore_architecture-2.png}
-caption[MyCoRe-Architecture and Components]{MyCoRe-Architecture and Componentsprotectfootnotemark}
-label{fig:abbMyCoReStruktur}</dd>
-</dl>
-<p>end{figure}
-footnotetext{source: <a class="reference external" href="https://www.mycore.de">https://www.mycore.de</a>}
-To specify the MyCoRe framework the morphilo application logic will have to be implemented,
-the TEI data model specified, and the input, search and output mask programmed.</p>
-<p>There are three directories which are
-important for adjusting the MyCoRe framework to the needs of one’s own application. These three directories
-correspond essentially to the three components in the MVC model as explicated in
-section ref{subsec:mvc}. Roughly, they are envisualized in figure ref{fig:abbMyCoReStruktur} in the upper
-right hand corner. More precisely, the view (emph{Layout} in figure ref{fig:abbMyCoReStruktur}) and the model layer
-(emph{Datenmodell} in figure ref{fig:abbMyCoReStruktur}) can be done
-completely via the <a href="#id1"><span class="problematic" id="id2">``</span></a>interface’‘, which is a directory with a predefined
-structure and some standard files. For the configuration of the logic an extra directory is offered (/src/main/java/custom/mycore/addons/). Here all, java classes
-extending the controller layer should be added.
-Practically, all three MVC layers are placed in the
-emph{src/main/}-directory of the application. In one of the subdirectories,
-emph{datamodel/def}, the datamodel specifications are defined as xml files. It parallels the model
-layer in the MVC pattern. How the data model was defined will be explained in
-section ref{subsec:datamodelimpl}.</p>
-</div>
-
-
- </div>
- </div>
- </div>
- <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
- <div class="sphinxsidebarwrapper"><div class="relations">
-<h3>Related Topics</h3>
-<ul>
- <li><a href="../index.html">Documentation overview</a><ul>
- <li>Previous: <a href="architecture.html" title="previous chapter">Software Design</a></li>
- </ul></li>
-</ul>
-</div>
- <div role="note" aria-label="source link">
- <h3>This Page</h3>
- <ul class="this-page-menu">
- <li><a href="../_sources/source/framework.rst.txt"
- rel="nofollow">Show Source</a></li>
- </ul>
- </div>
-<div id="searchbox" style="display: none" role="search">
- <h3>Quick search</h3>
- <div class="searchformwrapper">
- <form class="search" action="../search.html" method="get">
- <input type="text" name="q" />
- <input type="submit" value="Go" />
- <input type="hidden" name="check_keywords" value="yes" />
- <input type="hidden" name="area" value="default" />
- </form>
- </div>
-</div>
-<script type="text/javascript">$('#searchbox').show(0);</script>
- </div>
- </div>
- <div class="clearer"></div>
- </div>
- <div class="footer">
- ©2018, Hagen Peukert.
-
- |
- Powered by <a href="http://sphinx-doc.org/">Sphinx 1.7.2</a>
- & <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.10</a>
-
- |
- <a href="../_sources/source/framework.rst.txt"
- rel="nofollow">Page source</a>
- </div>
-
-
-
-
- </body>
-</html>
\ No newline at end of file
diff --git a/Morphilo_doc/_build/html/source/view.html b/Morphilo_doc/_build/html/source/view.html
index d9765a28c4f2be4ef9f21f8205467ef76c899ed4..aae493473e2a8fcee4aba91e13454b8a4e66c130 100644
--- a/Morphilo_doc/_build/html/source/view.html
+++ b/Morphilo_doc/_build/html/source/view.html
@@ -15,8 +15,8 @@
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="Software Design" href="architecture.html" />
- <link rel="prev" title="Controller Adjustments" href="controller.html" />
+ <link rel="next" title="Controller Adjustments" href="controller.html" />
+ <link rel="prev" title="Data Model" href="datamodel.html" />
<link rel="stylesheet" href="../_static/custom.css" type="text/css" />
@@ -35,340 +35,334 @@
<h1>View<a class="headerlink" href="#view" title="Permalink to this headline">¶</a></h1>
<div class="section" id="conceptualization">
<h2>Conceptualization<a class="headerlink" href="#conceptualization" title="Permalink to this headline">¶</a></h2>
-<p>Lastly, the third directory (emph{src/main/resources}) contains all code needed
+<p>The MyCoRe-directory (<em>src/main/resources</em>) contains all code needed
for rendering the data to be displayed on the screen. So this corresponds to
the view in an MVC approach. It is done by xsl-files that (unfortunately)
contain some logic that really belongs to the controller. Thus, the division is
-not as clear as implied in theory. I will discuss this issue more specifically in the
-relevant subsection below. Among the resources are also all images, styles, and
+not as clear as implied in theory. I will point at this issue more specifically in the
+relevant subsection below. Among the resources are all images, styles, and
javascripts.</p>
</div>
<div class="section" id="implementation">
<h2>Implementation<a class="headerlink" href="#implementation" title="Permalink to this headline">¶</a></h2>
-<p>As explained in section ref{subsec:mvc}, the view component handles the visual
+<p>The view component handles the visual
representation in the form of an interface that allows interaction between
the user and the task to be carried out by the machine. As a
webservice in the present case, all interaction happens via a browser, i.e. webpages are
visualized and responses are recognized by registering mouse or keyboard
events. More specifically, a webpage is rendered by transforming xml documents
to html pages. The MyCoRe repository framework uses an open source XSLT
-processor from Apache, Xalan.footnote{http://xalan.apache.org} This engine
+processor from Apache, <a class="reference external" href="http://xalan.apache.org">Xalan</a>. This engine
transforms document nodes described by the XPath syntax into hypertext making
use of a special form of template matching. All templates are collected in so
called xml-encoded stylesheets. Since there are two data models with two
different structures, it is good practice to define two stylesheet files one for
each data model.</p>
-<p>As a demonstration, in listing ref{lst:morphilostylesheet} below a short
+<p>As a demonstration, in the listing below a short
extract is given for rendering the word data.</p>
-<p>begin{lstlisting}[language=XML,caption={stylesheet
-morphilo.xsl},label=lst:morphilostylesheet]
-<?xml version=”1.0” encoding=”UTF-8”?>
-<xsl:stylesheet</p>
-<blockquote>
-<div><p>xmlns:xsl=”http://www.w3.org/1999/XSL/Transform”
-xmlns:xalan=”http://xml.apache.org/xalan”
-xmlns:i18n=”xalan://org.mycore.services.i18n.MCRTranslation”
-xmlns:acl=”xalan://org.mycore.access.MCRAccessManager”
-xmlns:mcr=”http://www.mycore.org/” xmlns:xlink=”http://www.w3.org/1999/xlink”
-xmlns:mods=”http://www.loc.gov/mods/v3”
-xmlns:encoder=”xalan://java.net.URLEncoder”
-xmlns:mcrxsl=”xalan://org.mycore.common.xml.MCRXMLFunctions”
-xmlns:mcrurn=”xalan://org.mycore.urn.MCRXMLFunctions”
-exclude-result-prefixes=”xalan xlink mcr i18n acl mods mcrxsl mcrurn encoder”
-version=”1.0”>
-<xsl:param name=”MCR.Users.Superuser.UserName”/></p>
-<dl class="docutils">
-<dt><xsl:template match=”/mycoreobject[contains(@ID,’_morphilo_’)]”></dt>
-<dd><dl class="first docutils">
-<dt><head></dt>
-<dd><link href=”{$WebApplicationBaseURL}css/file.css” rel=”stylesheet”/></dd>
-</dl>
-<p></head>
-<div class=”row”></p>
-<blockquote class="last">
-<div><dl class="docutils">
-<dt><xsl:call-template name=”objectAction”></dt>
-<dd><xsl:with-param name=”id” select=”@ID”/>
-<xsl:with-param name=”deriv” select=”structure/derobjects/derobject/@xlink:href”/></dd>
-</dl>
-<p></xsl:call-template>
-<xsl:variable name=”objID” select=”@ID”/>
-<!– Hier Ueberschrift setzen –>
-<h1 style=”text-indent: 4em;”></p>
-<blockquote>
-<div><dl class="docutils">
-<dt><xsl:if test=”metadata/def.morphiloContainer/morphiloContainer/morphilo/w”></dt>
-<dd><xsl:value-of select=”metadata/def.morphiloContainer/morphiloContainer/morphilo/w/text()[string-length(normalize-space(.))>0]”/></dd>
-</dl>
-<p></xsl:if></p>
-</div></blockquote>
-<p></h1>
-<dl class=”dl-horizontal”>
-<!– (1) Display word –></p>
-<blockquote>
-<div><dl class="docutils">
-<dt><xsl:if test=”metadata/def.morphiloContainer/morphiloContainer/morphilo/w”></dt>
-<dd><dl class="first docutils">
-<dt><dt></dt>
-<dd><xsl:value-of select=”i18n:translate(‘response.page.label.word’)”/></dd>
-</dl>
-<p></dt>
-<dd></p>
-<blockquote>
-<div><xsl:value-of select=”metadata/def.morphiloContainer/morphiloContainer/morphilo/w/text()[string-length(normalize-space(.))>0]”/></div></blockquote>
-<p class="last"></dd></p>
-</dd>
-</dl>
-<p></xsl:if></p>
-</div></blockquote>
-<dl class="docutils">
-<dt><!– (2) Display lemma –></dt>
-<dd>…</dd>
-</dl>
-</div></blockquote>
-</dd>
-</dl>
-<p></xsl:template>
-…
-<xsl:template name=”objectAction”>
-…
-</xsl:template></p>
-</div></blockquote>
-<p>…
-</xsl:stylesheet>
-end{lstlisting}
-This template matches with
-the root node of each emph{MyCoRe object} ensuring that a valid MyCoRe model is
+<div class="literal-block-wrapper docutils container" id="morphilo-xsl">
+<div class="code-block-caption"><span class="caption-text">word data rendering in morphilo.xsl</span><a class="headerlink" href="#morphilo-xsl" title="Permalink to this code">¶</a></div>
+<div class="highlight-xml notranslate"><div class="highlight"><pre><span></span><span class="cp"><?xml version="1.0" encoding="UTF-8"?></span>
+<span class="nt"><xsl:stylesheet</span>
+ <span class="na">xmlns:xsl=</span><span class="s">"http://www.w3.org/1999/XSL/Transform"</span>
+ <span class="na">xmlns:xalan=</span><span class="s">"http://xml.apache.org/xalan"</span>
+ <span class="na">xmlns:i18n=</span><span class="s">"xalan://org.mycore.services.i18n.MCRTranslation"</span>
+ <span class="na">xmlns:acl=</span><span class="s">"xalan://org.mycore.access.MCRAccessManager"</span>
+ <span class="na">xmlns:mcr=</span><span class="s">"http://www.mycore.org/"</span> <span class="na">xmlns:xlink=</span><span class="s">"http://www.w3.org/1999/xlink"</span>
+ <span class="na">xmlns:mods=</span><span class="s">"http://www.loc.gov/mods/v3"</span>
+ <span class="na">xmlns:encoder=</span><span class="s">"xalan://java.net.URLEncoder"</span>
+ <span class="na">xmlns:mcrxsl=</span><span class="s">"xalan://org.mycore.common.xml.MCRXMLFunctions"</span>
+ <span class="na">xmlns:mcrurn=</span><span class="s">"xalan://org.mycore.urn.MCRXMLFunctions"</span> <span class="na">exclude-result-prefixes=</span><span class="s">"xalan xlink mcr i18n acl mods mcrxsl mcrurn encoder"</span> <span class="na">version=</span><span class="s">"1.0"</span><span class="nt">></span>
+ <span class="nt"><xsl:param</span> <span class="na">name=</span><span class="s">"MCR.Users.Superuser.UserName"</span><span class="nt">/></span>
+ <span class="nt"><xsl:template</span> <span class="na">match=</span><span class="s">"/mycoreobject[contains(@ID,'_morphilo_')]"</span><span class="nt">></span>
+ <span class="nt"><head></span>
+ <span class="nt"><link</span> <span class="na">href=</span><span class="s">"{$WebApplicationBaseURL}css/file.css"</span> <span class="na">rel=</span><span class="s">"stylesheet"</span><span class="nt">/></span>
+ <span class="nt"></head></span>
+ <span class="nt"><div</span> <span class="na">class=</span><span class="s">"row"</span><span class="nt">></span>
+ <span class="nt"><xsl:call-template</span> <span class="na">name=</span><span class="s">"objectAction"</span><span class="nt">></span>
+ <span class="nt"><xsl:with-param</span> <span class="na">name=</span><span class="s">"id"</span> <span class="na">select=</span><span class="s">"@ID"</span><span class="nt">/></span>
+ <span class="nt"><xsl:with-param</span> <span class="na">name=</span><span class="s">"deriv"</span> <span class="na">select=</span><span class="s">"structure/derobjects/derobject/@xlink:href"</span><span class="nt">/></span>
+ <span class="nt"></xsl:call-template></span>
+ <span class="nt"><xsl:variable</span> <span class="na">name=</span><span class="s">"objID"</span> <span class="na">select=</span><span class="s">"@ID"</span><span class="nt">/></span>
+ <span class="c"><!-- Hier Ueberschrift setzen --></span>
+ <span class="nt"><h1</span> <span class="na">style=</span><span class="s">"text-indent: 4em;"</span><span class="nt">></span>
+ <span class="nt"><xsl:if</span> <span class="na">test=</span><span class="s">"metadata/def.morphiloContainer/morphiloContainer/morphilo/w"</span><span class="nt">></span>
+ <span class="nt"><xsl:value-of</span> <span class="na">select=</span><span class="s">"metadata/def.morphiloContainer/morphiloContainer/morphilo/w/text()[string-length(normalize-space(.))>0]"</span><span class="nt">/></span>
+ <span class="nt"></xsl:if></span>
+ <span class="nt"></h1></span>
+ <span class="nt"><dl</span> <span class="na">class=</span><span class="s">"dl-horizontal"</span><span class="nt">></span>
+ <span class="c"><!-- (1) Display word --></span>
+ <span class="nt"><xsl:if</span> <span class="na">test=</span><span class="s">"metadata/def.morphiloContainer/morphiloContainer/morphilo/w"</span><span class="nt">></span>
+ <span class="nt"><dt></span>
+ <span class="nt"><xsl:value-of</span> <span class="na">select=</span><span class="s">"i18n:translate('response.page.label.word')"</span><span class="nt">/></span>
+ <span class="nt"></dt></span>
+ <span class="nt"><dd></span>
+ <span class="nt"><xsl:value-of</span> <span class="na">select=</span><span class="s">"metadata/def.morphiloContainer/morphiloContainer/morphilo/w/text()[string-length(normalize-space(.))>0]"</span><span class="nt">/></span>
+ <span class="nt"></dd></span>
+ <span class="nt"></xsl:if></span>
+ <span class="c"><!-- (2) Display lemma --></span>
+ ...
+ <span class="nt"></xsl:template></span>
+ ...
+ <span class="nt"><xsl:template</span> <span class="na">name=</span><span class="s">"objectAction"</span><span class="nt">></span>
+ ...
+ <span class="nt"></xsl:template></span>
+...
+<span class="nt"></xsl:stylesheet></span>
+</pre></div>
+</div>
+</div>
+<p>This template matches with
+the root node of each <em>MyCoRe object</em> ensuring that a valid MyCoRe model is
used and checking that the document to be processed contains a unique
-identifier, here a emph{MyCoRe-ID}, and the name of the correct data model,
-here emph{morphilo}.
-Then, another template, emph{objectAction}, is called together with two parameters, the ids
+identifier, here a <em>MyCoRe-ID</em>, and the name of the correct data model,
+here <em>morphilo</em>.
+Then, another template, <em>objectAction</em>, is called together with two parameters, the ids
of the document object and attached files. In the remainder all relevant
information from the document is accessed by XPath, such as the word and the lemma,
and enriched with hypertext annotations it is rendered as a hypertext document.
-The template emph{objectAction} is key to understand the coupling process in the software
-framework. It is therefore separately listed in ref{lst:objActionTempl}.</p>
-<p>begin{lstlisting}[language=XML,caption={template
-objectAction},label=lst:objActionTempl,escapechar=|]
-<xsl:template name=”objectAction”></p>
-<blockquote>
-<div><p><xsl:param name=”id” select=”./@ID”/>
-<xsl:param name=”accessedit” select=”acl:checkPermission($id,’writedb’)”/>
-<xsl:param name=”accessdelete” select=”acl:checkPermission($id,’deletedb’)”/>
-<xsl:variable name=”derivCorp” select=”./@label”/>
-<xsl:variable name=”corpID” select=”metadata/def.corpuslink[@class=’MCRMetaLinkID’]/corpuslink/@xlink:href”/>
-<xsl:if test=”$accessedit or $accessdelete”>|label{ln:ng}|
-<div class=”dropdown pull-right”></p>
-<blockquote>
-<div><dl class="docutils">
-<dt><xsl:if test=”string-length($corpID) &gt; 0 or $CurrentUser=’administrator’”></dt>
-<dd><dl class="first docutils">
-<dt><button class=”btn btn-default dropdown-toggle” style=”margin:10px” type=”button” id=”dropdownMenu1” data-toggle=”dropdown” aria-expanded=”true”></dt>
-<dd><span class=”glyphicon glyphicon-cog” aria-hidden=”true”></span> Annotieren
-<span class=”caret”></span></dd>
-</dl>
-<p class="last"></button></p>
-</dd>
-</dl>
-<p></xsl:if>
-<xsl:if test=”string-length($corpID) &gt; 0”>|label{ln:ru}|</p>
-<blockquote>
-<div><p><xsl:variable name=”ifsDirectory” select=”document(concat(‘ifs:/’,$derivCorp))”/>
-<ul class=”dropdown-menu” role=”menu” aria-labelledby=”dropdownMenu1”></p>
-<blockquote>
-<div><dl class="docutils">
-<dt><li role=”presentation”></dt>
-<dd><dl class="first docutils">
-<dt><a href="#id1"><span class="problematic" id="id2">|\label{ln:nw1}|<a href="{$ServletsBaseURL}object/tag{$HttpSession}?id={$derivCorp}&amp;objID={$corpID}" role="menuitem" tabindex="-1">|</span></a>label{ln:nw2}|</dt>
-<dd><xsl:value-of select=”i18n:translate(‘object.nextObject’)”/></dd>
-</dl>
-<p class="last"></a></p>
-</dd>
-</dl>
-<p></li>
-<li role=”presentation”></p>
-<blockquote>
-<div><dl class="docutils">
-<dt><a href=”{$WebApplicationBaseURL}receive/{$corpID}” role=”menuitem” tabindex=”-1”></dt>
-<dd><xsl:value-of select=”i18n:translate(‘object.backToProject’)”/></dd>
-</dl>
-<p></a></p>
-</div></blockquote>
-<p></li></p>
-</div></blockquote>
-<p></ul></p>
-</div></blockquote>
-<p></xsl:if>
-<xsl:if test=”$CurrentUser=’administrator’”></p>
-<blockquote>
-<div><dl class="docutils">
-<dt><ul class=”dropdown-menu” role=”menu” aria-labelledby=”dropdownMenu1”></dt>
-<dd><blockquote class="first">
-<div><dl class="docutils">
-<dt><li role=”presentation”></dt>
-<dd><dl class="first docutils">
-<dt><a role=”menuitem” tabindex=”-1” href=”{$WebApplicationBaseURL}content/publish/morphilo.xed?id={$id}”></dt>
-<dd><xsl:value-of select=”i18n:translate(‘object.editWord’)”/></dd>
-</dl>
-<p class="last"></a></p>
-</dd>
-</dl>
-<p></li>
-<li role=”presentation”></p>
-<blockquote>
-<div><dl class="docutils">
-<dt><a href=”{$ServletsBaseURL}object/delete{$HttpSession}?id={$id}” role=”menuitem” tabindex=”-1” class=”confirm_deletion option” data-text=”Wirklich loeschen”></dt>
-<dd><xsl:value-of select=”i18n:translate(‘object.delWord’)”/></dd>
-</dl>
-<p></a></p>
-</div></blockquote>
-</div></blockquote>
-<p class="last"></li></p>
-</dd>
-</dl>
-<p></ul></p>
-</div></blockquote>
-<p></xsl:if>
-</div>
-<div class=”row” style=”margin-left:0px; margin-right:10px”></p>
-<blockquote>
-<div><dl class="docutils">
-<dt><xsl:apply-templates select=”structure/derobjects/derobject[acl:checkPermission(@xlink:href,’read’)]”></dt>
-<dd><xsl:with-param name=”objID” select=”@ID”/></dd>
-</dl>
-<p></xsl:apply-templates></p>
-</div></blockquote>
-<p></div></p>
-</div></blockquote>
-<p></xsl:if></p>
-</div></blockquote>
-<p></xsl:template>
-end{lstlisting}
-The emph{objectAction} template defines the selection menu appearing – once manual tagging has
+The template <em>objectAction</em> is key to understand the coupling process in the software
+framework. It is therefore separately listed in <a class="reference internal" href="#objactiontempl"><span class="std std-ref">template ObjectAction</span></a>.</p>
+<div class="literal-block-wrapper docutils container" id="id1">
+<span id="objactiontempl"></span><div class="code-block-caption"><span class="caption-text">template ObjectAction</span><a class="headerlink" href="#id1" title="Permalink to this code">¶</a></div>
+<div class="highlight-xml notranslate"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+10
+11
+12
+13
+14
+15
+16
+17
+18
+19
+20
+21
+22
+23
+24
+25
+26
+27
+28
+29
+30
+31
+32
+33
+34
+35
+36
+37
+38
+39
+40
+41
+42
+43
+44
+45
+46
+47
+48
+49
+50
+51</pre></div></td><td class="code"><div class="highlight"><pre><span></span><span class="nt"><xsl:template</span> <span class="na">name=</span><span class="s">"objectAction"</span><span class="nt">></span>
+<span class="nt"><xsl:param</span> <span class="na">name=</span><span class="s">"id"</span> <span class="na">select=</span><span class="s">"./@ID"</span><span class="nt">/></span>
+<span class="nt"><xsl:param</span> <span class="na">name=</span><span class="s">"accessedit"</span> <span class="na">select=</span><span class="s">"acl:checkPermission($id,'writedb')"</span><span class="nt">/></span>
+<span class="nt"><xsl:param</span> <span class="na">name=</span><span class="s">"accessdelete"</span> <span class="na">select=</span><span class="s">"acl:checkPermission($id,'deletedb')"</span><span class="nt">/></span>
+<span class="nt"><xsl:variable</span> <span class="na">name=</span><span class="s">"derivCorp"</span> <span class="na">select=</span><span class="s">"./@label"</span><span class="nt">/></span>
+<span class="nt"><xsl:variable</span> <span class="na">name=</span><span class="s">"corpID"</span> <span class="na">select=</span><span class="s">"metadata/def.corpuslink[@class='MCRMetaLinkID']/corpuslink/@xlink:href"</span><span class="nt">/></span>
+<span class="hll"><span class="nt"><xsl:if</span> <span class="na">test=</span><span class="s">"$accessedit or $accessdelete"</span><span class="nt">></span>
+</span><span class="nt"><div</span> <span class="na">class=</span><span class="s">"dropdown pull-right"</span><span class="nt">></span>
+ <span class="nt"><xsl:if</span> <span class="na">test=</span><span class="s">"string-length($corpID) &gt; 0 or $CurrentUser='administrator'"</span><span class="nt">></span>
+ <span class="nt"><button</span> <span class="na">class=</span><span class="s">"btn btn-default dropdown-toggle"</span> <span class="na">style=</span><span class="s">"margin:10px"</span> <span class="na">type=</span><span class="s">"button"</span> <span class="na">id=</span><span class="s">"dropdownMenu1"</span> <span class="na">data-toggle=</span><span class="s">"dropdown"</span> <span class="na">aria-expanded=</span><span class="s">"true"</span><span class="nt">></span>
+ <span class="nt"><span</span> <span class="na">class=</span><span class="s">"glyphicon glyphicon-cog"</span> <span class="na">aria-hidden=</span><span class="s">"true"</span><span class="nt">></span></span> Annotieren
+ <span class="nt"><span</span> <span class="na">class=</span><span class="s">"caret"</span><span class="nt">></span></span>
+ <span class="nt"></button></span>
+ <span class="nt"></xsl:if></span>
+<span class="hll"> <span class="nt"><xsl:if</span> <span class="na">test=</span><span class="s">"string-length($corpID) &gt; 0"</span><span class="nt">></span>
+</span> <span class="nt"><xsl:variable</span> <span class="na">name=</span><span class="s">"ifsDirectory"</span> <span class="na">select=</span><span class="s">"document(concat('ifs:/',$derivCorp))"</span><span class="nt">/></span>
+ <span class="nt"><ul</span> <span class="na">class=</span><span class="s">"dropdown-menu"</span> <span class="na">role=</span><span class="s">"menu"</span> <span class="na">aria-labelledby=</span><span class="s">"dropdownMenu1"</span><span class="nt">></span>
+ <span class="nt"><li</span> <span class="na">role=</span><span class="s">"presentation"</span><span class="nt">></span>
+<span class="hll"> <span class="nt"><a</span> <span class="na">href=</span><span class="s">"{$ServletsBaseURL}object/tag{$HttpSession}?id={$derivCorp}&amp;objID={$corpID}"</span> <span class="na">role=</span><span class="s">"menuitem"</span> <span class="na">tabindex=</span><span class="s">"-1"</span><span class="nt">></span>
+</span> <span class="nt"><xsl:value-of</span> <span class="na">select=</span><span class="s">"i18n:translate('object.nextObject')"</span><span class="nt">/></span>
+ <span class="nt"></a></span>
+ <span class="nt"></li></span>
+ <span class="nt"><li</span> <span class="na">role=</span><span class="s">"presentation"</span><span class="nt">></span>
+ <span class="nt"><a</span> <span class="na">href=</span><span class="s">"{$WebApplicationBaseURL}receive/{$corpID}"</span> <span class="na">role=</span><span class="s">"menuitem"</span> <span class="na">tabindex=</span><span class="s">"-1"</span><span class="nt">></span>
+ <span class="nt"><xsl:value-of</span> <span class="na">select=</span><span class="s">"i18n:translate('object.backToProject')"</span><span class="nt">/></span>
+ <span class="nt"></a></span>
+ <span class="nt"></li></span>
+ <span class="nt"></ul></span>
+ <span class="nt"></xsl:if></span>
+ <span class="nt"><xsl:if</span> <span class="na">test=</span><span class="s">"$CurrentUser='administrator'"</span><span class="nt">></span>
+ <span class="nt"><ul</span> <span class="na">class=</span><span class="s">"dropdown-menu"</span> <span class="na">role=</span><span class="s">"menu"</span> <span class="na">aria-labelledby=</span><span class="s">"dropdownMenu1"</span><span class="nt">></span>
+ <span class="nt"><li</span> <span class="na">role=</span><span class="s">"presentation"</span><span class="nt">></span>
+ <span class="nt"><a</span> <span class="na">role=</span><span class="s">"menuitem"</span> <span class="na">tabindex=</span><span class="s">"-1"</span> <span class="na">href=</span><span class="s">"{$WebApplicationBaseURL}content/publish/morphilo.xed?id={$id}"</span><span class="nt">></span>
+ <span class="nt"><xsl:value-of</span> <span class="na">select=</span><span class="s">"i18n:translate('object.editWord')"</span><span class="nt">/></span>
+ <span class="nt"></a></span>
+ <span class="nt"></li></span>
+ <span class="nt"><li</span> <span class="na">role=</span><span class="s">"presentation"</span><span class="nt">></span>
+ <span class="nt"><a</span> <span class="na">href=</span><span class="s">"{$ServletsBaseURL}object/delete{$HttpSession}?id={$id}"</span> <span class="na">role=</span><span class="s">"menuitem"</span> <span class="na">tabindex=</span><span class="s">"-1"</span> <span class="na">class=</span><span class="s">"confirm_deletion option"</span> <span class="na">data-text=</span><span class="s">"Wirklich loeschen"</span><span class="nt">></span>
+ <span class="nt"><xsl:value-of</span> <span class="na">select=</span><span class="s">"i18n:translate('object.delWord')"</span><span class="nt">/></span>
+ <span class="nt"></a></span>
+ <span class="nt"></li></span>
+ <span class="nt"></ul></span>
+ <span class="nt"></xsl:if></span>
+ <span class="nt"></div></span>
+ <span class="nt"><div</span> <span class="na">class=</span><span class="s">"row"</span> <span class="na">style=</span><span class="s">"margin-left:0px; margin-right:10px"</span><span class="nt">></span>
+ <span class="nt"><xsl:apply-templates</span> <span class="na">select=</span><span class="s">"structure/derobjects/derobject[acl:checkPermission(@xlink:href,'read')]"</span><span class="nt">></span>
+ <span class="nt"><xsl:with-param</span> <span class="na">name=</span><span class="s">"objID"</span> <span class="na">select=</span><span class="s">"@ID"</span><span class="nt">/></span>
+ <span class="nt"></xsl:apply-templates></span>
+ <span class="nt"></div></span>
+ <span class="nt"></xsl:if></span>
+ <span class="nt"></xsl:template></span>
+</pre></div>
+</td></tr></table></div>
+</div>
+<p>The <em>objectAction</em> template defines the selection menu appearing – once manual tagging has
started – on the upper right hand side of the webpage entitled
-emph{Annotieren} and displaying the two options emph{next word} or emph{back
-to project}.
-The first thing to note here is that in line ref{ln:ng} a simple test
+<em>Annotieren</em> and displaying the two options <em>next word</em> or <em>back
+to project</em>.
+The first thing to note here is that in line 7 a simple test
excludes all guest users from accessing the procedure. After ensuring that only
-the user who owns the corpus project has access (line ref{ln:ru}), s/he will be
+the user who owns the corpus project has access (line 15), s/he will be
able to access the drop down menu, which is really a url, e.g. line
-ref{ln:nw1}. The attentive reader might have noticed that
-the url exactly matches the definition in the web-fragment.xml as shown in
-listing ref{lst:webfragment}, line ref{ln:tag}, which resolves to the
+19. The attentive reader might have noticed that
+the url exactly matches the definition in the <em>web-fragment.xml</em> as shown in
+listing <a class="reference internal" href="controller.html#webxml"><span class="std std-ref">web-fragment.xml</span></a>, line 17, which resolves to the
respective java class there. Really, this mechanism is the data interface within the
-MVC pattern. The url also contains two variables, named emph{derivCorp} and
-emph{corpID}, that are needed to identify the corpus and file object by the
-java classes (see section ref{sec:javacode}).</p>
+MVC pattern. The url also contains two variables, named <em>derivCorp</em> and
+<em>corpID</em>, that are needed to identify the corpus and file object by the
+java classes (see section <a class="reference internal" href="controller.html#controller-section"><span class="std std-ref">Implementation</span></a>).</p>
<p>The morphilo.xsl stylesheet contains yet another modification that deserves mention.
-In listing ref{lst:derobjectTempl}, line ref{ln:morphMenu}, two menu options –
-emph{Tag automatically} and emph{Tag manually} – are defined. The former option
-initiates ProcessCorpusServlet.java as can be seen again in listing ref{lst:webfragment},
-line ref{ln:process}, which determines words that are not in the master data base.
+In listing <a class="reference internal" href="#derobjecttempl"><span class="std std-ref">derobject template</span></a>, line 18, two menu options –
+<em>Tag automatically</em> and <em>Tag manually</em> – are defined. The former option
+initiates ProcessCorpusServlet.java as can be seen again in listing <a class="reference internal" href="controller.html#webxml"><span class="std std-ref">web-fragment.xml</span></a>,
+line 7, which determines words that are not in the master data base.
Still, it is important to note that the menu option is only displayed if two restrictions
-are met. First, a file has to be uploaded (line ref{ln:1test}) and, second, there must be
+are met. First, a file has to be uploaded (line 19) and, second, there must be
only one file. This is necessary because in the annotation process other files will be generated
that store the words that were not yet processed or a file that includes the final result. The
generated files follow a certain pattern. The file harboring the final, entire TEI-annotated
-corpus is prefixed by emph{tagged}, the other file is prefixed emph{untagged}. This circumstance
-is exploited for manipulating the second option (line ref{ln:loop}). A loop runs through all
-files in the respective directory and if a file name starts with emph{untagged},
+corpus is prefixed by <em>tagged</em>, the other file is prefixed <em>untagged</em>. This circumstance
+is exploited for manipulating the second option (line 27). A loop runs through all
+files in the respective directory and if a file name starts with <em>untagged</em>,
the option to manually tag is displayed.</p>
-<p>begin{lstlisting}[language=XML,caption={template
-matching derobject},label=lst:derobjectTempl,escapechar=|]
-<xsl:template match=”derobject” mode=”derivateActions”></p>
-<blockquote>
-<div><p><xsl:param name=”deriv” />
-<xsl:param name=”parentObjID” />
-<xsl:param name=”suffix” select=”’‘” />
-<xsl:param name=”id” select=”../../../@ID” />
-<xsl:if test=”acl:checkPermission($deriv,’writedb’)”></p>
-<blockquote>
-<div><xsl:variable name=”ifsDirectory” select=”document(concat(‘ifs:’,$deriv,’/’))” />
-<xsl:variable name=”path” select=”$ifsDirectory/mcr_directory/path” /></div></blockquote>
-<dl class="docutils">
-<dt>…</dt>
-<dd><blockquote class="first">
-<div><dl class="docutils">
-<dt><div class=”options pull-right”></dt>
-<dd><dl class="first docutils">
-<dt><div class=”btn-group” style=”margin:10px”></dt>
-<dd><dl class="first docutils">
-<dt><a href=”#” class=”btn btn-default dropdown-toggle” data-toggle=”dropdown”></dt>
-<dd><i class=”fa fa-cog”></i>
-<xsl:value-of select=”’ Korpus’”/>
-<span class=”caret”></span></dd>
-</dl>
-<p class="last"></a></p>
-</dd>
-<dt><ul class=”dropdown-menu dropdown-menu-right”></dt>
-<dd><p class="first"><!– Anpasssungen Morphilo –>|label{ln:morphMenu}|
-<xsl:if test=”string-length($deriv) &gt; 0”>|label{ln:1test}|</p>
-<blockquote>
-<div><dl class="docutils">
-<dt><xsl:if test=”count($ifsDirectory/mcr_directory/children/child) = 1”>|label{ln:2test}|</dt>
-<dd><dl class="first docutils">
-<dt><li role=”presentation”></dt>
-<dd><dl class="first docutils">
-<dt><a href=”{$ServletsBaseURL}object/process{$HttpSession}?id={$deriv}&amp;objID={$id}” role=”menuitem” tabindex=”-1”></dt>
-<dd><xsl:value-of select=”i18n:translate(‘derivate.process’)”/></dd>
-</dl>
-<p class="last"></a></p>
-</dd>
-</dl>
-<p class="last"></li></p>
-</dd>
-</dl>
-<p></xsl:if>
-<xsl:for-each select=”$ifsDirectory/mcr_directory/children/child”>|label{ln:loop}|</p>
-<blockquote>
-<div><p><xsl:variable name=”untagged” select=”concat($path, ‘untagged’)”/>
-<xsl:variable name=”filename” select=”concat($path,./name)”/>
-<xsl:if test=”starts-with($filename, $untagged)”></p>
-<blockquote>
-<div><dl class="docutils">
-<dt><li role=”presentation”></dt>
-<dd><dl class="first docutils">
-<dt><a href=”{$ServletsBaseURL}object/tag{$HttpSession}?id={$deriv}&amp;objID={$id}” role=”menuitem” tabindex=”-1”></dt>
-<dd><xsl:value-of select=”i18n:translate(‘derivate.taggen’)”/></dd>
-</dl>
-<p class="last"></a></p>
-</dd>
-</dl>
-<p></li></p>
-</div></blockquote>
-<p></xsl:if></p>
-</div></blockquote>
-<p></xsl:for-each></p>
-</div></blockquote>
-<p class="last"></xsl:if></p>
-</dd>
-</dl>
-<p class="last">…
-</ul></p>
-</dd>
-</dl>
-<p></div></p>
-</div></blockquote>
-<p class="last"></div></p>
-</dd>
-</dl>
-<p></xsl:if></p>
-</div></blockquote>
-<p></xsl:template>
-end{lstlisting}</p>
-<p>Besides the two stylesheets morphilo.xsl and corpmeta.xsl, other stylesheets have
+<div class="literal-block-wrapper docutils container" id="id2">
+<span id="derobjecttempl"></span><div class="code-block-caption"><span class="caption-text">derobject template</span><a class="headerlink" href="#id2" title="Permalink to this code">¶</a></div>
+<div class="highlight-xml notranslate"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre> 1
+ 2
+ 3
+ 4
+ 5
+ 6
+ 7
+ 8
+ 9
+10
+11
+12
+13
+14
+15
+16
+17
+18
+19
+20
+21
+22
+23
+24
+25
+26
+27
+28
+29
+30
+31
+32
+33
+34
+35
+36
+37
+38
+39
+40
+41
+42
+43
+44</pre></div></td><td class="code"><div class="highlight"><pre><span></span><span class="nt"><xsl:template</span> <span class="na">match=</span><span class="s">"derobject"</span> <span class="na">mode=</span><span class="s">"derivateActions"</span><span class="nt">></span>
+ <span class="nt"><xsl:param</span> <span class="na">name=</span><span class="s">"deriv"</span> <span class="nt">/></span>
+ <span class="nt"><xsl:param</span> <span class="na">name=</span><span class="s">"parentObjID"</span> <span class="nt">/></span>
+ <span class="nt"><xsl:param</span> <span class="na">name=</span><span class="s">"suffix"</span> <span class="na">select=</span><span class="s">"''"</span> <span class="nt">/></span>
+ <span class="nt"><xsl:param</span> <span class="na">name=</span><span class="s">"id"</span> <span class="na">select=</span><span class="s">"../../../@ID"</span> <span class="nt">/></span>
+ <span class="nt"><xsl:if</span> <span class="na">test=</span><span class="s">"acl:checkPermission($deriv,'writedb')"</span><span class="nt">></span>
+ <span class="nt"><xsl:variable</span> <span class="na">name=</span><span class="s">"ifsDirectory"</span> <span class="na">select=</span><span class="s">"document(concat('ifs:',$deriv,'/'))"</span> <span class="nt">/></span>
+ <span class="nt"><xsl:variable</span> <span class="na">name=</span><span class="s">"path"</span> <span class="na">select=</span><span class="s">"$ifsDirectory/mcr_directory/path"</span> <span class="nt">/></span>
+ ...
+ <span class="nt"><div</span> <span class="na">class=</span><span class="s">"options pull-right"</span><span class="nt">></span>
+ <span class="nt"><div</span> <span class="na">class=</span><span class="s">"btn-group"</span> <span class="na">style=</span><span class="s">"margin:10px"</span><span class="nt">></span>
+ <span class="nt"><a</span> <span class="na">href=</span><span class="s">"#"</span> <span class="na">class=</span><span class="s">"btn btn-default dropdown-toggle"</span> <span class="na">data-toggle=</span><span class="s">"dropdown"</span><span class="nt">></span>
+ <span class="nt"><i</span> <span class="na">class=</span><span class="s">"fa fa-cog"</span><span class="nt">></i></span>
+ <span class="nt"><xsl:value-of</span> <span class="na">select=</span><span class="s">"' Korpus'"</span><span class="nt">/></span>
+ <span class="nt"><span</span> <span class="na">class=</span><span class="s">"caret"</span><span class="nt">></span></span>
+ <span class="nt"></a></span>
+ <span class="nt"><ul</span> <span class="na">class=</span><span class="s">"dropdown-menu dropdown-menu-right"</span><span class="nt">></span>
+<span class="hll"> <span class="c"><!-- Anpasssungen Morphilo --></span>|\label{ln:morphMenu}|
+</span><span class="hll"> <span class="nt"><xsl:if</span> <span class="na">test=</span><span class="s">"string-length($deriv) &gt; 0"</span><span class="nt">></span>|\label{ln:1test}|
+</span> <span class="nt"><xsl:if</span> <span class="na">test=</span><span class="s">"count($ifsDirectory/mcr_directory/children/child) = 1"</span><span class="nt">></span>|\label{ln:2test}|
+ <span class="nt"><li</span> <span class="na">role=</span><span class="s">"presentation"</span><span class="nt">></span>
+ <span class="nt"><a</span> <span class="na">href=</span><span class="s">"{$ServletsBaseURL}object/process{$HttpSession}?id={$deriv}&amp;objID={$id}"</span> <span class="na">role=</span><span class="s">"menuitem"</span> <span class="na">tabindex=</span><span class="s">"-1"</span><span class="nt">></span>
+ <span class="nt"><xsl:value-of</span> <span class="na">select=</span><span class="s">"i18n:translate('derivate.process')"</span><span class="nt">/></span>
+ <span class="nt"></a></span>
+ <span class="nt"></li></span>
+ <span class="nt"></xsl:if></span>
+<span class="hll"> <span class="nt"><xsl:for-each</span> <span class="na">select=</span><span class="s">"$ifsDirectory/mcr_directory/children/child"</span><span class="nt">></span>|\label{ln:loop}|
+</span> <span class="nt"><xsl:variable</span> <span class="na">name=</span><span class="s">"untagged"</span> <span class="na">select=</span><span class="s">"concat($path, 'untagged')"</span><span class="nt">/></span>
+ <span class="nt"><xsl:variable</span> <span class="na">name=</span><span class="s">"filename"</span> <span class="na">select=</span><span class="s">"concat($path,./name)"</span><span class="nt">/></span>
+ <span class="nt"><xsl:if</span> <span class="na">test=</span><span class="s">"starts-with($filename, $untagged)"</span><span class="nt">></span>
+ <span class="nt"><li</span> <span class="na">role=</span><span class="s">"presentation"</span><span class="nt">></span>
+ <span class="nt"><a</span> <span class="na">href=</span><span class="s">"{$ServletsBaseURL}object/tag{$HttpSession}?id={$deriv}&amp;objID={$id}"</span> <span class="na">role=</span><span class="s">"menuitem"</span> <span class="na">tabindex=</span><span class="s">"-1"</span><span class="nt">></span>
+ <span class="nt"><xsl:value-of</span> <span class="na">select=</span><span class="s">"i18n:translate('derivate.taggen')"</span><span class="nt">/></span>
+ <span class="nt"></a></span>
+ <span class="nt"></li></span>
+ <span class="nt"></xsl:if></span>
+ <span class="nt"></xsl:for-each></span>
+ <span class="nt"></xsl:if></span>
+ ...
+ <span class="nt"></ul></span>
+ <span class="nt"></div></span>
+ <span class="nt"></div></span>
+ <span class="nt"></xsl:if></span>
+<span class="nt"></xsl:template></span>
+</pre></div>
+</td></tr></table></div>
+</div>
+<p>Besides the two stylesheets <em>morphilo.xsl</em> and <em>corpmeta.xsl</em>, other stylesheets have
to be adjusted. They will not be discussed in detail here for they are self-explanatory for the most part.
-Essentially, they render the overall layout (emph{common-layout.xsl}, emph{skeleton_layout_template.xsl})
+Essentially, they render the overall layout (<em>common-layout.xsl</em>, <em>skeleton_layout_template.xsl</em>)
or the presentation
-of the search results (emph{response-page.xsl}) and definitions of the solr search fields (emph{searchfields-solr.xsl}).
-The former and latter also inherit templates from emph{response-general.xsl} and emph{response-browse.xsl}, in which the
+of the search results (<em>response-page.xsl</em>) and definitions of the solr search fields (<em>searchfields-solr.xsl</em>).
+The former and latter also inherit templates from <em>response-general.xsl</em> and <em>response-browse.xsl</em>, in which the
navigation bar of search results can be changed. For the use of multilinguality a separate configuration directory
-has to be created containing as many emph{.property}-files as different
-languages want to be displayed. In the current case these are restricted to German and English (emph{messages_de.properties} and emph{messages_en.properties}).
-The property files include all emph{i18n} definitions. All these files are located in the emph{resources} directory.</p>
+has to be created containing as many <em>.property</em>-files as different
+languages want to be displayed. In the current case these are restricted to German and English (<em>messages_de.properties</em> and <em>messages_en.properties</em>).
+The property files include all <em>i18n</em> definitions. All these files are located in the <em>resources</em> directory.</p>
<p>Furthermore, a search mask and a page for manually entering the annotations had
to be designed.
-For these files a specially designed xml standard (emph{xed}) is recommended to be used within the
+For these files a specially designed xml standard (<em>xed</em>) is recommended to be used within the
repository framework.</p>
</div>
</div>
@@ -391,8 +385,8 @@ repository framework.</p>
<h3>Related Topics</h3>
<ul>
<li><a href="../index.html">Documentation overview</a><ul>
- <li>Previous: <a href="controller.html" title="previous chapter">Controller Adjustments</a></li>
- <li>Next: <a href="architecture.html" title="next chapter">Software Design</a></li>
+ <li>Previous: <a href="datamodel.html" title="previous chapter">Data Model</a></li>
+ <li>Next: <a href="controller.html" title="next chapter">Controller Adjustments</a></li>
</ul></li>
</ul>
</div>
diff --git a/Morphilo_doc/_static/architecture.pdf b/Morphilo_doc/_static/architecture.pdf
deleted file mode 100644
index 2f671b5abd23245db733040b2e56873976a2f9fd..0000000000000000000000000000000000000000
Binary files a/Morphilo_doc/_static/architecture.pdf and /dev/null differ
diff --git a/Morphilo_doc/index.rst b/Morphilo_doc/index.rst
index c7043067b4c65dfe25c731ac0e2deb7221e09da4..466fbd75ad335baa0d368537c5a5e61d07652ab8 100644
--- a/Morphilo_doc/index.rst
+++ b/Morphilo_doc/index.rst
@@ -3,18 +3,18 @@
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
-Documentation Morphilo Project
-====================================
+Morphilo Project Documentation
+==============================
.. toctree::
:maxdepth: 3
:caption: Contents:
+ source/architecture.rst
source/datamodel.rst
- source/controller.rst
source/view.rst
- source/architecture.rst
- source/framework.rst
+ source/controller.rst
+
Indices and tables
==================
diff --git a/Morphilo_doc/source/architecture.rst b/Morphilo_doc/source/architecture.rst
index 5b114bda79b5dc2cca9d10df9d300507bf510317..4491bb3b056de35545bfdd78adb32b5d05afd31c 100644
--- a/Morphilo_doc/source/architecture.rst
+++ b/Morphilo_doc/source/architecture.rst
@@ -1,56 +1,95 @@
Software Design
===============
+MVC Model
+---------
-.. image:: architecture.*
+A standard architecture for software has become a form of an
+observer pattern called *Model-View-Controller (MVC)*-Model [#f3]_.
+This is escpecially true for web-based applications that use
+some form of a client-server architecture since these systems naturally divide
+the browser view from the rest of the program logic and, if dynamically set up,
+also from the data model usually running in an extra server as well.
+As already implied, the MVC-pattern modularizes the program into three components: model, view, and
+controller coupled *low* by interfaces. The view is concerned with
+everything the actual user sees on the screen or uses to interact with the
+machine. The controller is to recognize and process the events initiated by the
+user and to update the view. Processing involves to communicate with the model.
+This may involve to save or provide data from the data base.
+From all that follows, MVC-models are especially supportive for reusing
+existing software and promotes parallel development of its three components.
+So the data model of an existing program can easily be changed without touching
+the essentials of the program logic. The same is true for the code that handles
+the view. Most of the time view and data model are the two components that need
+to be changed so that the software appearance and presentation is adjusted to
+the new user group as well as the different data is adjusted to the needs of the different
+requirements of the new application. Nevertheless, if bugs or general changes in
+the controller component have to be done, it usually does not affect
+substantially the view and data model.
-The architecture of a possible **take-and-share**-approach for language
-resources is visualized in figure \ref{fig:architect}. Because the very gist
+Another positive consequence of MVC-models is that several views (or even
+models) could be used simultaneously. It means that the same data could be
+presented differently on the user interface.
+
+
+Morphilo Architecture
+---------------------
+
+
+.. figure:: images/architecture.png
+
+ Figure 1: Basic Architecture of a Take-&-Share-Approach
+
+
+The architecture of a possible *take-and-share* approach for language
+resources is visualized in figure 1. Because the very gist
of the approach becomes clearer if describing a concrete example, the case of
-annotating lexical derivatives of Middle English and a respective database is
-given as an illustration.
+annotating lexical derivatives of Middle English with the help of the Morphilo Tool
+[#f1]_ using a `MyCoRe repository <http://www.mycore.de>`_ is given as an illustration.
However, any other tool that helps with manual annotations and manages metadata of a corpus could be
-substituted here instead.
+substituted here instead. [#f2]_
After inputting an untagged corpus or plain text, it is determined whether the
input material was annotated previously by a different user. This information is
usually provided by the metadata administered by the annotation tool; in the case at
-hand it is called \emph{Morphilizer} in figure \ref{fig:architect}. An
-alternative is a simple table look-up for all occurring words in the datasets Corpus 1 through Corpus n. If contained
-completely, the \emph{yes}-branch is followed up further -- otherwise \emph{no}
+hand, the *Morphilo* component. An alternative is a
+simple table look-up for all occurring words in the datasets Corpus 1 through Corpus n. If contained
+completely, the *yes*-branch is followed up further -- otherwise *no*
succeeds. The difference between the two branches is subtle, yet crucial. On
-both branches, the annotation tool (here \emph{Morphilizer}) is called, which, first,
-sorts out all words that are not contained in the master database (here \emph{Morphilo-DB})
+both branches, the annotation tool (here *Morphilo*) is called, which, first,
+sorts out all words that are not contained in the master database (here *MyCoRe* repository)
and, second, makes reasonable suggestions on an optimal annotation of
-the items. In both cases the
-annotations are linked to the respective items (e.g. words) in the
-text, but they are also persistently saved in an extra dataset, i.e. Corpus 1
+the items. The suggestions made to the user are based on simple string mapping of a saved list of prefixes and suffixes
+whereas the remainder of the mapping is defined as the word root. The annotations are linked to the respective items (e.g. words) in the
+text, but they are also persistently saved in an extra dataset, i.e. in figure 1 in one of the delineated Corpus 1
through n, together with all available metadata.
-The difference between both information streams is that
-in the \emph{yes}-branch a comparison between the newly created dataset and
-all of the previous datasets of this text is carried out. Within this
-unit, all deviations and congruencies are marked and counted. The underlying
+The difference between the two branches in figure 1 is that
+in the *yes*-branch a comparison between the newly created dataset and
+all of the previous datasets of this text is carried out while this is not
+possible if a text was not annotated before. Within this
+unit, all deviations and congruencies of the annotated items are marked and counted. The underlying
assumption is that with a growing number of comparable texts the
correct annotations approach a theoretic true value of a correct annotation
while errors level out provided that the sample size is large enough. How the
distribution of errors and correct annotations exactly looks like and if a
normal distribution can be assumed is still object of the ongoing research, but
-independent of the concrete results, the component (called \emph{compare
-manual annotations} in figure \ref{fig:architect}) allows for specifying the
+independent of the concrete results, the component (called *compare
+manual annotations* in figure 1) allows for specifying the
exact form of the sample population.
In fact, it is necessary at that point to define the form of the distribution,
-sample size, and the rejection region. The standard setting are a normal
-distribution, a rejection region of $\alpha = 0.05$ and sample size of $30$ so
-that a simple Gau\ss-Test can be calculated.
+sample size, and the rejection region. To be put it simple here, a uniform distribution in form of a threshold value
+of e.g. 20 could be defined that specifies that a word has to be annotated equally by
+20 different users before it enters the master database.
-Continuing the information flow further, these statistical calculations are
+Continuing the information flow in figure 1 further, the threshold values or, if so defined,
+the results of the statistical calculation of other distributions respectively are
delivered to the quality-control-component. Based on the statistics, the
respective items together with the metadata, frequencies, and, of course,
annotations are written to the master database. All information in the master
database is directly used for automated annotations. Thus it is directly matched
-to the input texts or corpora respectively through the \emph{Morphilizer}-tool.
+to the input texts or corpora respectively through the *Morphilo*-tool.
The annotation tool decides on the entries looked up in the master which items
are to be manually annotated.
@@ -63,4 +102,35 @@ possible to interrupt the annotation work and save progress on the server. And
the user will have access to the annotations made in the respective dataset,
correct them or save them and resume later. It is important to note that the user will receive
the tagged document only after all items are fully annotated. No partially
-tagged text can be output.
\ No newline at end of file
+tagged text can be output.
+
+Repository Framework
+--------------------
+
+.. figure:: images/mycore_architecture-2.png
+
+ Figure 2: `MyCoRe <http://www.mycore.de>`_-Architecture and Components
+
+To specify the repository framework, the morphilo application logic will have to be implemented,
+a data model specified, and the input, search and output mask programmed.
+
+There are three directories which are
+important for adjusting the MyCoRe framework to the needs of one's own application.
+
+These three directories
+correspond essentially to the three components in the MVC model as explicated above. Roughly, they are also envisualized in figure 2 in the upper
+right hand corner. More precisely, the view (*Layout* in figure 2) and the model layer
+(*Datenmodell* in figure 2) can be done
+completely via the *interface*, which is a directory with a predefined
+structure and some standard files. For the configuration of the logic an extra directory is offered (*/src/main/java/custom/mycore/addons/*). Here all, java classes
+extending the controller layer should be added.
+Practically, all three MVC layers are placed in the
+*src/main/*-directory of the application. In one of the subdirectories,
+*datamodel/def,* the datamodel specifications are defined as xml files. It parallels the model
+layer in the MVC pattern. How the data model was defined will be explained in the section Data Model.
+
+.. rubric:: Notes
+
+.. [#f1] Peukert, H. (2012): From Semi-Automatic to Automatic Affix Extraction in Middle English Corpora: Building a Sustainable Database for Analyzing Derivational Morphology over Time, Empirical Methods in Natural Language Processing, Wien, Scientific series of the ÖGAI, 413-23.
+.. [#f2] The source code of a possible implementation is available on https://github.com/amadeusgwin/morphilo. The software runs in test mode on https://www.morphilo.uni-hamburg.de/content/index.xml.
+.. [#f3] Butz, Andreas; Antonio Krüger (2017): Mensch-Maschine-Interaktion, De Gruyter, pp. 93.
\ No newline at end of file
diff --git a/Morphilo_doc/source/controller.rst b/Morphilo_doc/source/controller.rst
index 6f6b896272e6cade94e54ff34fd76af6bfc28326..e63e0e81c05dd42f0aed235e5467e316d70ca96e 100644
--- a/Morphilo_doc/source/controller.rst
+++ b/Morphilo_doc/source/controller.rst
@@ -4,11 +4,18 @@ Controller Adjustments
General Principle of Operation
------------------------------
-Figure \ref{fig:classDiag} illustrates the dependencies of the five java classes that were integrated to add the morphilo
-functionality defined in the default package \emph{custom.mycore.addons.morphilo}. The general principle of operation
+.. _classDiag:
+
+.. figure:: images/morphilo_uml.png
+
+ Figure 3: Morphilo UML Diagramm
+
+
+Figure :ref:`classDiag` illustrates the dependencies of the five java classes that were integrated to add the morphilo
+functionality defined in the default package *custom.mycore.addons.morphilo*. The general principle of operation
is the following. The handling of data search, upload, saving, and user
authentification is fully left to the MyCoRe functionality that is completely
-implemented. The class \emph{ProcessCorpusServlet.java} receives a request from the webinterface to process an uploaded file,
+implemented. The class *ProcessCorpusServlet.java* receives a request from the webinterface to process an uploaded file,
i.e. a simple text corpus, and it checks if any of the words are available in the master database. All words that are not
listed in the master database are written to an extra file. These are the words that have to be manually annotated. At the end, the
servlet sends a response back to the user interface. In case of all words are contained in the master, an xml file is generated from the
@@ -16,38 +23,29 @@ master database that includes all annotated words of the original corpus. Usuall
So if some words are not in the master, the user will get the response to initiate the manual annotation process.
The manual annotation process is processed by the class
-\emph{{Tag\-Corpus\-Serv\-let\-.ja\-va}}, which will build a JDOM object for the first word in the extra file.
-This is done by creating an object of the \emph{JDOMorphilo.java} class. This class, in turn, will use the methods of
-\emph{AffixStripper.java} that make simple, but reasonable, suggestions on the word structure. This JDOM object is then
+*TagCorpusServlet.java*, which will build a JDOM object for the first word in the extra file.
+This is done by creating an object of the *JDOMorphilo.java* class. This class, in turn, will use the methods of
+*AffixStripper.java* that make simple, but reasonable, suggestions on the word structure. This JDOM object is then
given as a response back to the user. It is presented as a form, in which the user can make changes. This is necessary
-because the word structure algorithm of \emph{AffixStripper.java} errs in some cases. Once the user agrees on the
+because the word structure algorithm of *AffixStripper.java* errs in some cases. Once the user agrees on the
suggestions or on his or her corrections, the JDOM object is saved as an xml that is only searchable, visible, and
changeable by the authenicated user (and the administrator), another file containing all processed words is created or
-updated respectively and the \emph{TagCorpusServlet.java} servlet will restart until the last word in the extra list is
+updated respectively and the *TagCorpusServlet.java* servlet will restart until the last word in the extra list is
processed. This enables the user to stop and resume her or his annotation work at a later point in time. The
-\emph{TagCorpusServlet} will call methods from \emph{ProcessCorpusServlet.java} to adjust the content of the extra
+*TagCorpusServlet* will call methods from *ProcessCorpusServlet.java* to adjust the content of the extra
files harboring the untagged words. If this file is empty, and only then, it is replaced by the file comprising all words
from the original text file, both the ones from the master database and the ones that are annotated by the user,
in an annotated xml representation.
-Each time \emph{ProcessCorpusServlet.java} is instantiated, it also instantiates \emph{QualityControl.java}. This class checks if a
+Each time *ProcessCorpusServlet.java* is instantiated, it also instantiates *QualityControl.java*. This class checks if a
new word can be transferred to the master database. The algorithm can be freely adopted to higher or lower quality standards.
In its present configuration, a method tests at a limit of 20 different
registered users agreeing on the annotation of the same word. More specifically,
-if 20 JDOM objects are identical except in the attribute field \emph{occurrences} in the metadata node, the JDOM object becomes
-part of the master. The latter is easily done by changing the attribute \emph{creator} from the user name
-to \emph{``administrator''} in the service node. This makes the dataset part of the master database. Moreover, the \emph{occurrences}
+if 20 JDOM objects are identical except in the attribute field *occurrences* in the metadata node, the JDOM object becomes
+part of the master. The latter is easily done by changing the attribute *creator* from the user name
+to *administrator* in the service node. This makes the dataset part of the master database. Moreover, the *occurrences*
attribute is updated by adding up all occurrences of the word that stem from
different text corpora of the same time range.
-\begin{landscape}
- \begin{figure}
- \centering
- \includegraphics[scale=0.55]{morphilo_uml.png}
- \caption{Class Diagram Morphilo}
- \label{fig:classDiag}
- \end{figure}
-\end{landscape}
-
Conceptualization
@@ -60,533 +58,572 @@ files as saving, creating, deleting, and updating files.
Moreover, a rudimentary user management comprising different roles and
rights is offered. The basic technology behind the controller's logic is the
servlet. As such all new code has to be registered as a servlet in the
-web-fragment.xml (here the Apache Tomcat container) as listing \ref{lst:webfragment} shows.
-
-\begin{lstlisting}[language=XML,caption={Servlet Registering in the
-web-fragment.xml (excerpt)},label=lst:webfragment,escapechar=|]
-<servlet>
- <servlet-name>ProcessCorpusServlet</servlet-name>
- <servlet-class>custom.mycore.addons.morphilo.ProcessCorpusServlet</servlet-class>
-</servlet>
-<servlet-mapping>
- <servlet-name>ProcessCorpusServlet</servlet-name>
- <url-pattern>/servlets/object/process</url-pattern>|\label{ln:process}|
-</servlet-mapping>
-<servlet>
- <servlet-name>TagCorpusServlet</servlet-name>
- <servlet-class>custom.mycore.addons.morphilo.TagCorpusServlet</servlet-class>
-</servlet>
-<servlet-mapping>
- <servlet-name>TagCorpusServlet</servlet-name>
- <url-pattern>/servlets/object/tag</url-pattern>|\label{ln:tag}|
-</servlet-mapping>
-\end{lstlisting}
-
-Now, the logic has to be extended by the specifications analyzed in chapter
-\ref{chap:concept} on conceptualization. More specifically, some
+web-fragment.xml (here the Apache Tomcat container).
+
+.. _webxml:
+
+.. code-block:: xml
+ :caption: Servlet Registering in the web-fragment.xml
+ :linenos:
+ :emphasize-lines: 7, 15
+
+ <servlet>
+ <servlet-name>ProcessCorpusServlet</servlet-name>
+ <servlet-class>custom.mycore.addons.morphilo.ProcessCorpusServlet</servlet-class>
+ </servlet>
+ <servlet-mapping>
+ <servlet-name>ProcessCorpusServlet</servlet-name>
+ <url-pattern>/servlets/object/process</url-pattern>
+ </servlet-mapping>
+ <servlet>
+ <servlet-name>TagCorpusServlet</servlet-name>
+ <servlet-class>custom.mycore.addons.morphilo.TagCorpusServlet</servlet-class>
+ </servlet>
+ <servlet-mapping>
+ <servlet-name>TagCorpusServlet</servlet-name>
+ <url-pattern>/servlets/object/tag</url-pattern>
+ </servlet-mapping>
+ \end{lstlisting}
+
+Now, the logic has to be extended by the specifications. Some
classes have to be added that take care of analyzing words
-(\emph{AffixStripper.java, InflectionEnum.java, SuffixEnum.java,
-PrefixEnum.java}), extracting the relevant words from the text and checking the
-uniqueness of the text (\emph{ProcessCorpusServlet.java}), make reasonable
-suggestions on the annotation (\emph{TagCorpusServlet.java}), build the object
-of each annotated word (\emph{JDOMorphilo.java}), and check on the quality by applying
-statistical models (\emph{QualityControl.java}).
+(*AffixStripper.java, InflectionEnum.java, SuffixEnum.java,
+PrefixEnum.java*), extracting the relevant words from the text and checking the
+uniqueness of the text (*ProcessCorpusServlet.java*), make reasonable
+suggestions on the annotation (*TagCorpusServlet.java*), build the object
+of each annotated word (*JDOMorphilo.java*), and check on the quality by applying
+statistical models (*QualityControl.java*).
+
+.. _controller-section:
Implementation
--------------
Having taken a bird's eye perspective in the previous chapter, it is now time to take a look at the specific implementation at the level
-of methods. Starting with the main servlet, \emph{ProcessCorpusServlet.java}, the class defines four getter method:
-\renewcommand{\labelenumi}{(\theenumi)}
-\begin{enumerate}
- \item\label{itm:geturl} public String getURLParameter(MCRServletJob, String)
- \item\label{itm:getcorp} public String getCorpusMetadata(MCRServletJob, String)
- \item\label{itm:getcont} public ArrayList<String> getContentFromFile(MCRServletJob, String)
- \item\label{itm:getderiv} public Path getDerivateFilePath(MCRServletJob, String)
- \item\label{itm:now} public int getNumberOfWords(MCRServletJob job, String)
-\end{enumerate}
+of methods. Starting with the main servlet, *ProcessCorpusServlet.java*, the class defines four getter method:
+
+#. *public String getURLParameter(MCRServletJob, String)*
+#. *public String getCorpusMetadata(MCRServletJob, String)*
+#. *public ArrayList<String> getContentFromFile(MCRServletJob, String)*
+#. *public Path getDerivateFilePath(MCRServletJob, String)*
+#. *public int getNumberOfWords(MCRServletJob job, String)*
+
Since each servlet in MyCoRe extends the class MCRServlet, it has access to MCRServletJob, from which the http requests and responses
-can be used. This is the first argument in the above methods. The second argument of method (\ref{itm:geturl}) specifies the name of an url parameter, i.e.
-the object id or the id of the derivate. The method returns the value of the given parameter. Typically MyCoRe uses the url to exchange
+can be used. This is the first argument in the above methods. The second argument of method (in 1.) specifies the name of an url parameter, i.e.
+the object id or the id of the derivate. The method returns the value of the given parameter. Typically *MyCoRe* uses the url to exchange
these ids. The second method provides us with the value of a data field in the xml document. So the string defines the name of an attribute.
-\emph{getContentFromFile(MCRServletJob, String)} returns the words as a list from a file when given the filename as a string.
-The getter listed in \ref{itm:getderiv}), returns the Path from the MyCoRe repository when the name of
-the file is specified. And finally, method (\ref{itm:now}) returns the number of words by simply returning
-\emph{getContentFromFile(job, fileName).size()}.
+*getContentFromFile(MCRServletJob, String)* returns the words as a list from a file when given the filename as a string.
+The getter listed in 4., returns the Path from the MyCoRe repository when the name of
+the file is specified. And finally, method (in 5.) returns the number of words by simply returning
+*getContentFromFile(job, fileName).size()*.
There are two methods in every MyCoRe-Servlet that have to be overwritten,
-\emph{protected void render(MCRServletJob, Exception)}, which redirects the requests as \emph{POST} or \emph{GET} responds, and
-\emph{protected void think(MCRServletJob)}, in which the logic is implemented. Since the latter is important to understand the
-core idea of the Morphilo algorithm, it is displayed in full length in source code \ref{src:think}.
-
-\begin{lstlisting}[language=java,caption={The overwritten think method},label=src:think,escapechar=|]
-protected void think(MCRServletJob job) throws Exception
-{
- this.job = job;
- String dateFromCorp = getCorpusMetadata(job, "def.datefrom");
- String dateUntilCorp = getCorpusMetadata(job, "def.dateuntil");
- String corpID = getURLParameter(job, "objID");
- String derivID = getURLParameter(job, "id");
-
- //if NoW is 0, fill with anzWords
- MCRObject helpObj = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(corpID));|\label{ln:bugfixstart}|
- Document jdomDocHelp = helpObj.createXML();
- XPathFactory xpfacty = XPathFactory.instance();
- XPathExpression<Element> xpExp = xpfacty.compile("//NoW", Filters.element());
- Element elem = xpExp.evaluateFirst(jdomDocHelp);
- //fixes transferred morphilo data from previous stand alone project
- int corpussize = getNumberOfWords(job, "");
- if (Integer.parseInt(elem.getText()) != corpussize)
- {
- elem.setText(Integer.toString(corpussize));
- helpObj = new MCRObject(jdomDocHelp);
- MCRMetadataManager.update(helpObj);
- }|\label{ln:bugfixend}|
-
- //Check if the uploaded corpus was processed before
- SolrClient slr = MCRSolrClientFactory.getSolrClient();|\label{ln:solrstart}|
- SolrQuery qry = new SolrQuery();
- qry.setFields("korpusname", "datefrom", "dateuntil", "NoW", "id");
- qry.setQuery("datefrom:" + dateFromCorp + " AND dateuntil:" + dateUntilCorp + " AND NoW:" + corpussize);
- SolrDocumentList rslt = slr.query(qry).getResults();|\label{ln:solrresult}|
-
- Boolean incrOcc = true;
- // if resultset contains only one, then it must be the newly created corpus
- if (slr.query(qry).getResults().getNumFound() > 1)
- {
- incrOcc = false;
- }|\label{ln:solrend}|
-
- //match all words in corpus with morphilo (creator=administrator) and save all words that are not in morphilo DB in leftovers
- ArrayList<String> leftovers = new ArrayList<String>();
- ArrayList<String> processed = new ArrayList<String>();
-
- leftovers = getUnknownWords(getContentFromFile(job, ""), dateFromCorp, dateUntilCorp, "", incrOcc, incrOcc, false);|\label{ln:callkeymeth}|
-
- //write all words of leftover in file as derivative to respective corpmeta dataset
- MCRPath root = MCRPath.getPath(derivID, "/");|\label{ln:filesavestart}|
- Path fn = getDerivateFilePath(job, "").getFileName();
- Path p = root.resolve("untagged-" + fn);
- Files.write(p, leftovers);|\label{ln:filesaveend}|
-
- //create a file for all words that were processed
- Path procWds = root.resolve("processed-" + fn);
- Files.write(procWds, processed);
-}
-\end{lstlisting}
-Using the above mentioned getter methods, the \emph{think} method assigns values to the object ID, needed to get the xml document
-that contain the corpus metadata, the file ID, and the beginning and starting dates from the corpus to be analyzed. Lines \ref{ln:bugfixstart}
-through \ref{ln:bugfixend} show how to access a mycore object as an xml document, a procedure that will be used in different variants
+*protected void render(MCRServletJob, Exception)*, which redirects the requests as *POST* or *GET* responds, and
+*protected void think(MCRServletJob)*, in which the logic is implemented. Since the latter is important to understand the
+core idea of the Morphilo algorithm, it is displayed in full length in source code :ref:`think`.
+
+.. _think:
+
+.. code-block:: java
+ :caption: The overwritten think method
+ :linenos:
+ :emphasize-lines: 10, 22, 25, 36, 42, 45, 48
+
+ protected void think(MCRServletJob job) throws Exception
+ {
+ this.job = job;
+ String dateFromCorp = getCorpusMetadata(job, "def.datefrom");
+ String dateUntilCorp = getCorpusMetadata(job, "def.dateuntil");
+ String corpID = getURLParameter(job, "objID");
+ String derivID = getURLParameter(job, "id");
+
+ //if NoW is 0, fill with anzWords
+ MCRObject helpObj = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(corpID));
+ Document jdomDocHelp = helpObj.createXML();
+ XPathFactory xpfacty = XPathFactory.instance();
+ XPathExpression<Element> xpExp = xpfacty.compile("//NoW", Filters.element());
+ Element elem = xpExp.evaluateFirst(jdomDocHelp);
+ //fixes transferred morphilo data from previous stand alone project
+ int corpussize = getNumberOfWords(job, "");
+ if (Integer.parseInt(elem.getText()) != corpussize)
+ {
+ elem.setText(Integer.toString(corpussize));
+ helpObj = new MCRObject(jdomDocHelp);
+ MCRMetadataManager.update(helpObj);
+ }
+
+ //Check if the uploaded corpus was processed before
+ SolrClient slr = MCRSolrClientFactory.getSolrClient();
+ SolrQuery qry = new SolrQuery();
+ qry.setFields("korpusname", "datefrom", "dateuntil", "NoW", "id");
+ qry.setQuery("datefrom:" + dateFromCorp + " AND dateuntil:" + dateUntilCorp + " AND NoW:" + corpussize);
+ SolrDocumentList rslt = slr.query(qry).getResults();
+
+ Boolean incrOcc = true;
+ // if resultset contains only one, then it must be the newly created corpus
+ if (slr.query(qry).getResults().getNumFound() > 1)
+ {
+ incrOcc = false;
+ }
+
+ //match all words in corpus with morphilo (creator=administrator) and save all words that are not in morphilo DB in leftovers
+ ArrayList<String> leftovers = new ArrayList<String>();
+ ArrayList<String> processed = new ArrayList<String>();
+
+ leftovers = getUnknownWords(getContentFromFile(job, ""), dateFromCorp, dateUntilCorp, "", incrOcc, incrOcc, false);
+
+ //write all words of leftover in file as derivative to respective corpmeta dataset
+ MCRPath root = MCRPath.getPath(derivID, "/");
+ Path fn = getDerivateFilePath(job, "").getFileName();
+ Path p = root.resolve("untagged-" + fn);
+ Files.write(p, leftovers);
+
+ //create a file for all words that were processed
+ Path procWds = root.resolve("processed-" + fn);
+ Files.write(procWds, processed);
+ }
+
+Using the above mentioned getter methods, the *think*-method assigns values to the object ID, needed to get the xml document
+that contains the corpus metadata, the file ID, and the beginning and starting dates from the corpus to be analyzed. Lines 10
+through 22 show how to access a mycore object as an xml document, a procedure that will be used in different variants
throughout this implementation.
By means of the object ID, the respective corpus is identified and a JDOM document is constructed, which can then be accessed
by XPath. The XPath factory instances are collections of the xml nodes. In the present case, it is save to assume that only one element
-of \emph{NoW} is available (see corpus datamodel listing \ref{lst:corpusdatamodel} with $maxOccurs='1'$). So we do not have to loop through
-the collection, but use the first node named \emph{NoW}. The if-test checks if the number of words of the uploaded file is the
+of *NoW* is available (see corpus datamodel listing :ref:`corpusdatamodel` with *maxOccurs='1'*). So we do not have to loop through
+the collection, but use the first node named *NoW*. The if-test checks if the number of words of the uploaded file is the
same as the number written in the document. When the document is initially created by the MyCoRe logic it was configured to be zero.
-If unequal, the setText(String) method is used to write the number of words of the corpus to the document.
+If unequal, the *setText(String)* method is used to write the number of words of the corpus to the document.
-Lines \ref{ln:solrstart}--\ref{ln:solrend} reveal the second important ingredient, i.e. controlling the search engine. First, a solr
+Lines 25--36 reveal the second important ingredient, i.e. controlling the search engine. First, a solr
client and a query are initialized. Then, the output of the result set is defined by giving the fields of interest of the document.
-In the case at hand, it is the id, the name of the corpus, the number of words, and the beginnig and ending dates. With \emph{setQuery}
-it is possible to assign values to some or all of these fields. Finally, \emph{getResults()} carries out the search and writes
-all hits to a \emph{SolrDocumentList} (line \ref{ln:solrresult}). The test that follows is really only to set a Boolean
+In the case at hand, it is the id, the name of the corpus, the number of words, and the beginnig and ending dates. With *setQuery*
+it is possible to assign values to some or all of these fields. Finally, *getResults()* carries out the search and writes
+all hits to a *SolrDocumentList* (line 29). The test that follows is really only to set a Boolean
encoding if the number of occurrences of that word in the master should be updated. To avoid multiple counts,
incrementing the word frequency is only done if it is a new corpus.
-In line \ref{ln:callkeymeth} \emph{getUnknownWords(ArrayList, String, String, String, Boolean, Boolean, Boolean)} is called and
+In line 42 *getUnknownWords(ArrayList, String, String, String, Boolean, Boolean, Boolean)* is called and
returned as a list of words. This method is key and will be discussed in depth below. Finally, lines
-\ref{ln:filesavestart}--\ref{ln:filesaveend} show how to handle file objects in MyCoRe. Using the file ID, the root path and the name
-of the first file in that path are identified. Then, a second file starting with ``untagged'' is created and all words returned from
-the \emph{getUnknownWords} is written to that file. By the same token an empty file is created (in the last two lines of the \emph{think}-method),
+45--48 show how to handle file objects in MyCoRe. Using the file ID, the root path and the name
+of the first file in that path are identified. Then, a second file starting with *untagged* is created and all words returned from
+the *getUnknownWords* is written to that file. By the same token an empty file is created (in the last two lines of the *think*-method),
in which all words that are manually annotated will be saved.
-In a refactoring phase, the method \emph{getUnknownWords(ArrayList, String, String, String, Boolean, Boolean, Boolean)} could be subdivided into
+In a refactoring phase, the method *getUnknownWords(ArrayList, String, String, String, Boolean, Boolean, Boolean)* could be subdivided into
three methods: for each Boolean parameter one. In fact, this method handles more than one task. This is mainly due to multiple code avoidance.
-%this is just wrong because no resultset will substantially be more than 10-20
-%In addition, for large text files this method would run into efficiency problems if the master database also reaches the intended size of about
-%$100,000$ entries and beyond because
In essence, an outer loop runs through all words of the corpus and an inner loop runs through all hits in the solr result set. Because the result
-set is supposed to be small, approximately between $10-20$ items, efficiency
+set is supposed to be small, approximately between 10-20 items, efficiency
problems are unlikely to cause a problem, although there are some more loops running through collection of about the same sizes.
-%As the hits naturally grow larger with an increasing size of the data base, processing time will rise exponentially.
Since each word is identified on the basis of its projected word type, the word form, and the time range it falls into, it is these variables that
have to be checked for existence in the documents. If not in the xml documents,
-\emph{null} is returned and needs to be corrected. Moreover, user authentification must be considered. There are three different XPaths that are relevant.
-\begin{itemize}
- \item[-] \emph{//service/servflags/servflag[@type='createdby']} to test for the correct user
- \item[-] \emph{//morphiloContainer/morphilo} to create the annotated document
- \item[-] \emph{//morphiloContainer/morphilo/w} to set occurrences or add a link
-\end{itemize}
-
-As an illustration of the core functioning of this method, listing \ref{src:getUnknowWords} is given.
-\begin{lstlisting}[language=java,caption={Mode of Operation of getUnknownWords Method},label=src:getUnknowWords,escapechar=|]
-public ArrayList<String> getUnknownWords(
- ArrayList<String> corpus,
- String timeCorpusBegin,
- String timeCorpusEnd,
- String wdtpe,
- Boolean setOcc,
- Boolean setXlink,
- Boolean writeAllData) throws Exception
- {
- String currentUser = MCRSessionMgr.getCurrentSession().getUserInformation().getUserID();
- ArrayList lo = new ArrayList();
-
- for (int i = 0; i < corpus.size(); i++)
- {
- SolrClient solrClient = MCRSolrClientFactory.getSolrClient();
- SolrQuery query = new SolrQuery();
- query.setFields("w","occurrence","begin","end", "id", "wordtype");
- query.setQuery(corpus.get(i));
- query.setRows(50); //more than 50 items are extremely unlikely
- SolrDocumentList results = solrClient.query(query).getResults();
- Boolean available = false;
- for (int entryNum = 0; entryNum < results.size(); entryNum++)
- {
- ...
- // update in MCRMetaDataManager
- String mcrIDString = results.get(entryNum).getFieldValue("id").toString();
- //MCRObjekt auslesen und JDOM-Document erzeugen:
- MCRObject mcrObj = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(mcrIDString));
- Document jdomDoc = mcrObj.createXML();
- ...
- //check and correction for word type
- ...
- //checkand correction time: timeCorrect
- ...
- //check if user correct: isAuthorized
- ...
- XPathExpression<Element> xp = xpfac.compile("//morphiloContainer/morphilo/w", Filters.element());
- //Iterates w-elements and increments occurrence attribute if setOcc is true
- for (Element e : xp.evaluate(jdomDoc))
+*null* is returned and needs to be corrected. Moreover, user authentification must be considered. There are three different XPaths that are relevant.
+
+* *//service/servflags/servflag[@type='createdby']* to test for the correct user
+* *//morphiloContainer/morphilo* to create the annotated document
+* *//morphiloContainer/morphilo/w* to set occurrences or add a link
+
+As an illustration of the core functioning of this method, listing :ref:`getUnknowWords` is given.
+
+.. _getUnknowWords:
+
+.. code-block:: java
+ :caption: Mode of Operation of getUnknownWords Method
+ :linenos:
+ :emphasize-lines: 47, 60, 67, 69, 77
+
+ public ArrayList<String> getUnknownWords(
+ ArrayList<String> corpus,
+ String timeCorpusBegin,
+ String timeCorpusEnd,
+ String wdtpe,
+ Boolean setOcc,
+ Boolean setXlink,
+ Boolean writeAllData) throws Exception
{
- //wenn Rechte da sind und Worttyp nirgends gegeben oder gleich ist
- if (isAuthorized && timeCorrect
- && ((e.getAttributeValue("wordtype") == null && wdtpe.equals(""))
- || e.getAttributeValue("wordtype").equals(wordtype))) // nur zur Vereinheitlichung
+ String currentUser = MCRSessionMgr.getCurrentSession().getUserInformation().getUserID();
+ ArrayList lo = new ArrayList();
+
+ for (int i = 0; i < corpus.size(); i++)
+ {
+ SolrClient solrClient = MCRSolrClientFactory.getSolrClient();
+ SolrQuery query = new SolrQuery();
+ query.setFields("w","occurrence","begin","end", "id", "wordtype");
+ query.setQuery(corpus.get(i));
+ query.setRows(50); //more than 50 items are extremely unlikely
+ SolrDocumentList results = solrClient.query(query).getResults();
+ Boolean available = false;
+ for (int entryNum = 0; entryNum < results.size(); entryNum++)
{
- int oc = -1;
- available = true;|\label{ln:available}|
- try
- {
- //adjust occurrence Attribut
- if (setOcc)
+ ...
+ // update in MCRMetaDataManager
+ String mcrIDString = results.get(entryNum).getFieldValue("id").toString();
+ //MCRObjekt auslesen und JDOM-Document erzeugen:
+ MCRObject mcrObj = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(mcrIDString));
+ Document jdomDoc = mcrObj.createXML();
+ ...
+ //check and correction for word type
+ ...
+ //checkand correction time: timeCorrect
+ ...
+ //check if user correct: isAuthorized
+ ...
+ XPathExpression<Element> xp = xpfac.compile("//morphiloContainer/morphilo/w", Filters.element());
+ //Iterates w-elements and increments occurrence attribute if setOcc is true
+ for (Element e : xp.evaluate(jdomDoc))
+ {
+ //wenn Rechte da sind und Worttyp nirgends gegeben oder gleich ist
+ if (isAuthorized && timeCorrect
+ && ((e.getAttributeValue("wordtype") == null && wdtpe.equals(""))
+ || e.getAttributeValue("wordtype").equals(wordtype))) // nur zur Vereinheitlichung
{
- oc = Integer.parseInt(e.getAttributeValue("occurrence"));
- e.setAttribute("occurrence", Integer.toString(oc + 1));
- }
-
- //write morphilo-ObjectID in xml of corpmeta
- if (setXlink)
- {
- Namespace xlinkNamespace = Namespace.getNamespace("xlink", "http://www.w3.org/1999/xlink");|\label{ln:namespace}|
- MCRObject corpObj = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(getURLParameter(job, "objID")));
- Document corpDoc = corpObj.createXML();
- XPathExpression<Element> xpathEx = xpfac.compile("//corpuslink", Filters.element());
- Element elm = xpathEx.evaluateFirst(corpDoc);
- elm.setAttribute("href" , mcrIDString, xlinkNamespace);
- }
- mcrObj = new MCRObject(jdomDoc);|\label{ln:updatestart}|
- MCRMetadataManager.update(mcrObj);
- QualityControl qc = new QualityControl(mcrObj);|\label{ln:updateend}|
- }
- catch(NumberFormatException except)
- {
- // ignore
- }
- }
- }
- if (!available) // if not available in datasets under the given conditions |\label{ln:notavailable}|
- {
- lo.add(corpus.get(i));
- }
- }
- return lo;
- }
-\end{lstlisting}
-As can be seen from the functionality of listing \ref{src:getUnknowWords}, getting the unknown words of a corpus, is rather a side effect for the equally named method.
-More precisely, a Boolean (line \ref{ln:available}) is set when the document is manipulated otherwise because it is clear that the word must exist then.
-If the Boolean remains false (line \ref{ln:notavailable}), the word is put on the list of words that have to be annotated manually. As already explained above, the
+ int oc = -1;
+ available = true;
+ try
+ {
+ //adjust occurrence Attribut
+ if (setOcc)
+ {
+ oc = Integer.parseInt(e.getAttributeValue("occurrence"));
+ e.setAttribute("occurrence", Integer.toString(oc + 1));
+ }
+
+ //write morphilo-ObjectID in xml of corpmeta
+ if (setXlink)
+ {
+ Namespace xlinkNamespace = Namespace.getNamespace("xlink", "http://www.w3.org/1999/xlink");
+ MCRObject corpObj = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(getURLParameter(job, "objID")));
+ Document corpDoc = corpObj.createXML();
+ XPathExpression<Element> xpathEx = xpfac.compile("//corpuslink", Filters.element());
+ Element elm = xpathEx.evaluateFirst(corpDoc);
+ elm.setAttribute("href" , mcrIDString, xlinkNamespace);
+ }
+ mcrObj = new MCRObject(jdomDoc);
+ MCRMetadataManager.update(mcrObj);
+ QualityControl qc = new QualityControl(mcrObj);
+ }
+ catch(NumberFormatException except)
+ {
+ // ignore
+ }
+ }
+ }
+ if (!available) // if not available in datasets under the given conditions
+ {
+ lo.add(corpus.get(i));
+ }
+ }
+ return lo;
+ }
+
+As can be seen from the functionality of listing :ref:`getUnknowWords`, getting the unknown words of a corpus, is rather a side effect for the equally named method.
+More precisely, a Boolean (line 47) is set when the document is manipulated otherwise because it is clear that the word must exist then.
+If the Boolean remains false (line 77), the word is put on the list of words that have to be annotated manually. As already explained above, the
first loop runs through all words (corpus) and the following lines a solr result set is created. This set is also looped through and it is checked if the time range,
-the word type and the user are authorized. In the remainder, the occurrence attribute of the morphilo document can be incremented (setOcc is true) or/and the word is linked to the
-corpus meta data (setXlink is true). While all code lines are equivalent with
-what was explained in listing \ref{src:think}, it suffices to focus on an
+the word type and the user are authorized. In the remainder, the occurrence attribute of the morphilo document can be incremented (*setOcc* is true) or/and the word is linked to the
+corpus meta data (*setXlink* is true). While all code lines are equivalent with
+what was explained in listing :ref:`think`, it suffices to focus on an
additional name space, i.e.
-``xlink'' has to be defined (line \ref{ln:namespace}). Once the linking of word
-and corpus is set, the entire MyCoRe object has to be updated. This is done by the functionality of the framework (lines \ref{ln:updatestart}--\ref{ln:updateend}).
-At the end, an instance of \emph{QualityControl} is created.
-
-%QualityControl
-The class \emph{QualityControl} is instantiated with a constructor
-depicted in listing \ref{src:constructQC}.
-\begin{lstlisting}[language=java,caption={Constructor of QualityControl.java},label=src:constructQC,escapechar=|]
-private MCRObject mycoreObject;
-/* Constructor calls method to carry out quality control, i.e. if at least 20
- * different users agree 100% on the segments of the word under investigation
- */
-public QualityControl(MCRObject mycoreObject) throws Exception
-{
- this.mycoreObject = mycoreObject;
- if (getEqualObjectNumber() > 20)
- {
- addToMorphiloDB();
- }
-}
-\end{lstlisting}
+*xlink* has to be defined (line 60). Once the linking of word
+and corpus is set, the entire MyCoRe object has to be updated. This is done by the functionality of the framework (lines 67--69).
+At the end, an instance of *QualityControl* is created.
+
+The class *QualityControl* is instantiated with a constructor
+depicted in listing :ref:`constructQC`.
+
+.. _constructQC:
+
+.. code-block:: java
+ :caption: Constructor of QualityControl.java
+
+ private MCRObject mycoreObject;
+ /* Constructor calls method to carry out quality control, i.e. if at least 20
+ * different users agree 100% on the segments of the word under investigation
+ */
+ public QualityControl(MCRObject mycoreObject) throws Exception
+ {
+ this.mycoreObject = mycoreObject;
+ if (getEqualObjectNumber() > 20)
+ {
+ addToMorphiloDB();
+ }
+ }
+
The constructor takes an MyCoRe object, a potential word candidate for the
master data base, which is assigned to a private class variable because the
object is used though not changed by some other java methods.
-More importantly, there are two more methods: \emph{getEqualNumber()} and
-\emph{addToMorphiloDB()}. While the former initiates a process of counting and
+More importantly, there are two more methods: *getEqualNumber()* and
+*addToMorphiloDB()*. While the former initiates a process of counting and
comparing objects, the latter is concerned with calculating the correct number
-of occurrences from different, but not the same texts, and generating a MyCoRe object with the same content but with two different flags in the \emph{//service/servflags/servflag}-node, i.e. \emph{createdby='administrator'} and \emph{state='published'}.
-And of course, the \emph{occurrence} attribute is set to the newly calculated value. The logic corresponds exactly to what was explained in
-listing \ref{src:think} and will not be repeated here. The only difference are the paths compiled by the XPathFactory. They are
-\begin{itemize}
- \item[-] \emph{//service/servflags/servflag[@type='createdby']} and
- \item[-] \emph{//service/servstates/servstate[@classid='state']}.
-\end{itemize}
+of occurrences from different, but not the same texts, and generating a MyCoRe object with the same
+content but with two different flags in the *//service/servflags/servflag*-node, i.e. *createdby='administrator'* and *state='published'*.
+And of course, the *occurrence* attribute is set to the newly calculated value. The logic corresponds exactly to what was explained in
+listing :ref:`think` and will not be repeated here. The only difference are the paths compiled by the XPathFactory. They are
+
+* *//service/servflags/servflag[@type='createdby']* and
+* *//service/servstates/servstate[@classid='state']*.
+
It is more instructive to document how the number of occurrences is calculated. There are two steps involved. First, a list with all mycore objects that are
-equal to the object which the class is instantiated with (``mycoreObject'' in listing \ref{src:constructQC}) is created. This list is looped and all occurrence
+equal to the object which the class is instantiated with (*mycoreObject* in listing :ref:`constructQC`) is created. This list is looped and all occurrence
attributes are summed up. Second, all occurrences from equal texts are substracted. Equal texts are identified on the basis of its meta data and its derivate.
-There are some obvious shortcomings of this approach, which will be discussed in chapter \ref{chap:results}, section \ref{sec:improv}. Here, suffice it to
-understand the mode of operation. Listing \ref{src:equalOcc} shows a possible solution.
-\begin{lstlisting}[language=java,caption={Occurrence Extraction from Equal Texts (1)},label=src:equalOcc,escapechar=|]
-/* returns number of Occurrences if Objects are equal, zero otherwise
- */
-private int getOccurrencesFromEqualTexts(MCRObject mcrobj1, MCRObject mcrobj2) throws SAXException, IOException
-{
- int occurrences = 1;
- //extract corpmeta ObjectIDs from morphilo-Objects
- String crpID1 = getAttributeValue("//corpuslink", "href", mcrobj1);
- String crpID2 = getAttributeValue("//corpuslink", "href", mcrobj2);
- //get these two corpmeta Objects
- MCRObject corpo1 = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(crpID1));
- MCRObject corpo2 = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(crpID2));
- //are the texts equal? get list of 'processed-words' derivate
- String corp1DerivID = getAttributeValue("//structure/derobjects/derobject", "href", corpo1);
- String corp2DerivID = getAttributeValue("//structure/derobjects/derobject", "href", corpo2);
-
- ArrayList result = new ArrayList(getContentFromFile(corp1DerivID, ""));|\label{ln:writeContent}|
- result.remove(getContentFromFile(corp2DerivID, ""));|\label{ln:removeContent}|
- if (result.size() == 0) // the texts are equal
- {
- // extract occurrences of one the objects
- occurrences = Integer.parseInt(getAttributeValue("//morphiloContainer/morphilo/w", "occurrence", mcrobj1));
- }
- else
- {
- occurrences = 0; //project metadata happened to be the same, but texts are different
- }
- return occurrences;
-}
-\end{lstlisting}
-In this implementation, the ids from the \emph{corpmeta} data model are accessed via the xlink attribute in the morphilo documents.
-The method \emph{getAttributeValue(String, String, MCRObject)} does exactly the same as demonstrated earlier (see from line \ref{ln:namespace}
-on in listing \ref{src:getUnknowWords}). The underlying logic is that the texts are equal if exactly the same number of words were uploaded.
-So all words from one file are written to a list (line \ref{ln:writeContent}) and words from the other file are removed from the
-very same list (line \ref{ln:removeContent}). If this list is empty, then the exact same number of words must have been in both files and the occurrences
-are adjusted accordingly. Since this method is called from another private method that only contains a loop through all equal objects, one gets
-the occurrences from all equal texts. For reasons of confirmability, the looping method is also given:
-\begin{lstlisting}[language=java,caption={Occurrence Extraction from Equal Texts (2)},label=src:equalOcc2,escapechar=|]
-private int getOccurrencesFromEqualTexts() throws Exception
-{
- ArrayList<MCRObject> equalObjects = new ArrayList<MCRObject>();
- equalObjects = getAllEqualMCRObjects();
- int occurrences = 0;
- for (MCRObject obj : equalObjects)
- {
- occurrences = occurrences + getOccurrencesFromEqualTexts(mycoreObject, obj);
- }
- return occurrences;
-}
-\end{lstlisting}
-
-Now, the constructor in listing \ref{src:constructQC} reveals another method that rolls out an equally complex concatenation of procedures.
-As implied above, \emph{getEqualObjectNumber()} returns the number of equally annotated words. It does this by falling back to another
-method from which the size of the returned list is calculated (\emph{getAllEqualMCRObjects().size()}). Hence, we should care about
-\emph{getAllEqualMCRObjects()}. This method really has the same design as \emph{int getOccurrencesFromEqualTexts()} in listing \ref{src:equalOcc2}.
-The difference is that another method (\emph{Boolean compareMCRObjects(MCRObject, MCRObject, String)}) is used within the loop and
-that all equal objects are put into the list of MyCoRe objects that are returned. If this list comprises more than 20
-entries,\footnote{This number is somewhat arbitrary. It is inspired by the sample size n in t-distributed data.} the respective document
-will be integrated in the master data base by the process described above.
-The comparator logic is shown in listing \ref{src:compareMCR}.
-\begin{lstlisting}[language=java,caption={Comparison of MyCoRe objects},label=src:compareMCR,escapechar=|]
-private Boolean compareMCRObjects(MCRObject mcrobj1, MCRObject mcrobj2, String xpath) throws SAXException, IOException
-{
- Boolean isEqual = false;
- Boolean beginTime = false;
- Boolean endTime = false;
- Boolean occDiff = false;
- Boolean corpusDiff = false;
-
- String source = getXMLFromObject(mcrobj1, xpath);
- String target = getXMLFromObject(mcrobj2, xpath);
-
- XMLUnit.setIgnoreAttributeOrder(true);
- XMLUnit.setIgnoreComments(true);
- XMLUnit.setIgnoreDiffBetweenTextAndCDATA(true);
- XMLUnit.setIgnoreWhitespace(true);
- XMLUnit.setNormalizeWhitespace(true);
-
- //differences in occurrences, end, begin should be ignored
- try
- {
- Diff xmlDiff = new Diff(source, target);
- DetailedDiff dd = new DetailedDiff(xmlDiff);
- //counters for differences
- int i = 0;
- int j = 0;
- int k = 0;
- int l = 0;
- // list containing all differences
- List differences = dd.getAllDifferences();|\label{ln:difflist}|
- for (Object object : differences)
+
+.. code-block:: java
+ :caption: Occurrence Extraction from Equal Texts
+ :linenos:
+
+ /* returns number of Occurrences if Objects are equal, zero otherwise
+ */
+ private int getOccurrencesFromEqualTexts(MCRObject mcrobj1, MCRObject mcrobj2) throws SAXException, IOException
{
- Difference difference = (Difference) object;
- //@begin,@end,... node is not in the difference list if the count is 0
- if (difference.getControlNodeDetail().getXpathLocation().endsWith("@begin")) i++;|\label{ln:diffbegin}|
- if (difference.getControlNodeDetail().getXpathLocation().endsWith("@end")) j++;
- if (difference.getControlNodeDetail().getXpathLocation().endsWith("@occurrence")) k++;
- if (difference.getControlNodeDetail().getXpathLocation().endsWith("@corpus")) l++;|\label{ln:diffend}|
- //@begin and @end have different values: they must be checked if they fall right in the allowed time range
- if ( difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())
- && difference.getControlNodeDetail().getXpathLocation().endsWith("@begin")
- && (Integer.parseInt(difference.getControlNodeDetail().getValue()) < Integer.parseInt(difference.getTestNodeDetail().getValue())) )
+ int occurrences = 1;
+ //extract corpmeta ObjectIDs from morphilo-Objects
+ String crpID1 = getAttributeValue("//corpuslink", "href", mcrobj1);
+ String crpID2 = getAttributeValue("//corpuslink", "href", mcrobj2);
+ //get these two corpmeta Objects
+ MCRObject corpo1 = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(crpID1));
+ MCRObject corpo2 = MCRMetadataManager.retrieveMCRObject(MCRObjectID.getInstance(crpID2));
+ //are the texts equal? get list of 'processed-words' derivate
+ String corp1DerivID = getAttributeValue("//structure/derobjects/derobject", "href", corpo1);
+ String corp2DerivID = getAttributeValue("//structure/derobjects/derobject", "href", corpo2);
+
+ ArrayList result = new ArrayList(getContentFromFile(corp1DerivID, ""));
+ result.remove(getContentFromFile(corp2DerivID, ""));
+ if (result.size() == 0) // the texts are equal
{
- beginTime = true;
+ // extract occurrences of one the objects
+ occurrences = Integer.parseInt(getAttributeValue("//morphiloContainer/morphilo/w", "occurrence", mcrobj1));
}
- if (difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())
- && difference.getControlNodeDetail().getXpathLocation().endsWith("@end")
- && (Integer.parseInt(difference.getControlNodeDetail().getValue()) > Integer.parseInt(difference.getTestNodeDetail().getValue())) )
+ else
{
- endTime = true;
+ occurrences = 0; //project metadata happened to be the same, but texts are different
}
- //attribute values of @occurrence and @corpus are ignored if they are different
- if (difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())
- && difference.getControlNodeDetail().getXpathLocation().endsWith("@occurrence"))
+ return occurrences;
+ }
+
+In this implementation, the ids from the *corpmeta* data model are accessed via the xlink attribute in the morphilo documents.
+The method *getAttributeValue(String, String, MCRObject)* does exactly the same as demonstrated earlier (see from line 60
+on in listing :ref:`getUnknowWords`). The underlying logic is that the texts are equal if exactly the same number of words were uploaded.
+So all words from one file are written to a list (line 16) and words from the other file are removed from the
+very same list (line 17). If this list is empty, then the exact same number of words must have been in both files and the occurrences
+are adjusted accordingly. Since this method is called from another private method that only contains a loop through all equal objects, one gets
+the occurrences from all equal texts. For reasons of confirmability, the looping method is also given:
+
+.. _equalOcc2:
+
+.. code-block:: java
+ :caption: Occurrence Extraction from Equal Texts (2)
+ :linenos:
+
+ private int getOccurrencesFromEqualTexts() throws Exception
+ {
+ ArrayList<MCRObject> equalObjects = new ArrayList<MCRObject>();
+ equalObjects = getAllEqualMCRObjects();
+ int occurrences = 0;
+ for (MCRObject obj : equalObjects)
{
- occDiff = true;
+ occurrences = occurrences + getOccurrencesFromEqualTexts(mycoreObject, obj);
}
- if (difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())
- && difference.getControlNodeDetail().getXpathLocation().endsWith("@corpus"))
- {
- corpusDiff = true;
- }
+ return occurrences;
}
- //if any of @begin, @end ... is identical set Boolean to true
- if (i == 0) beginTime = true;|\label{ln:zerobegin}|
- if (j == 0) endTime = true;
- if (k == 0) occDiff = true;
- if (l == 0) corpusDiff = true;|\label{ln:zeroend}|
- //if the size of differences is greater than the number of changes admitted in @begin, @end ... something else must be different
- if (beginTime && endTime && occDiff && corpusDiff && (i + j + k + l) == dd.getAllDifferences().size()) isEqual = true;|\label{ln:diffsum}|
- }
- catch (SAXException e)
- {
- e.printStackTrace();
- }
- catch (IOException e)
+
+
+Now, the constructor in listing :ref:`constructQC` reveals another method that rolls out an equally complex concatenation of procedures.
+As implied above, *getEqualObjectNumber()* returns the number of equally annotated words. It does this by falling back to another
+method from which the size of the returned list is calculated (*getAllEqualMCRObjects().size()*). Hence, we should care about
+*getAllEqualMCRObjects()*. This method really has the same design as *int getOccurrencesFromEqualTexts()* in listing :ref:`equalOcc2`.
+The difference is that another method (*Boolean compareMCRObjects(MCRObject, MCRObject, String)*) is used within the loop and
+that all equal objects are put into the list of MyCoRe objects that are returned. If this list comprises more than 20
+entries, [#f4]_ the respective document
+will be integrated in the master data base by the process described above.
+The comparator logic is shown in listing :ref:`compareMCR`.
+
+.. _compareMCR:
+
+.. code-block:: java
+ :caption: Comparison of MyCoRe objects
+ :linenos:
+ :emphasize-lines: 29, 34, 37, 64, 67, 69
+
+ private Boolean compareMCRObjects(MCRObject mcrobj1, MCRObject mcrobj2, String xpath) throws SAXException, IOException
{
- e.printStackTrace();
+ Boolean isEqual = false;
+ Boolean beginTime = false;
+ Boolean endTime = false;
+ Boolean occDiff = false;
+ Boolean corpusDiff = false;
+
+ String source = getXMLFromObject(mcrobj1, xpath);
+ String target = getXMLFromObject(mcrobj2, xpath);
+
+ XMLUnit.setIgnoreAttributeOrder(true);
+ XMLUnit.setIgnoreComments(true);
+ XMLUnit.setIgnoreDiffBetweenTextAndCDATA(true);
+ XMLUnit.setIgnoreWhitespace(true);
+ XMLUnit.setNormalizeWhitespace(true);
+
+ //differences in occurrences, end, begin should be ignored
+ try
+ {
+ Diff xmlDiff = new Diff(source, target);
+ DetailedDiff dd = new DetailedDiff(xmlDiff);
+ //counters for differences
+ int i = 0;
+ int j = 0;
+ int k = 0;
+ int l = 0;
+ // list containing all differences
+ List differences = dd.getAllDifferences();
+ for (Object object : differences)
+ {
+ Difference difference = (Difference) object;
+ //@begin,@end,... node is not in the difference list if the count is 0
+ if (difference.getControlNodeDetail().getXpathLocation().endsWith("@begin")) i++;
+ if (difference.getControlNodeDetail().getXpathLocation().endsWith("@end")) j++;
+ if (difference.getControlNodeDetail().getXpathLocation().endsWith("@occurrence")) k++;
+ if (difference.getControlNodeDetail().getXpathLocation().endsWith("@corpus")) l++;
+ //@begin and @end have different values: they must be checked if they fall right in the allowed time range
+ if ( difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())
+ && difference.getControlNodeDetail().getXpathLocation().endsWith("@begin")
+ && (Integer.parseInt(difference.getControlNodeDetail().getValue()) < Integer.parseInt(difference.getTestNodeDetail().getValue())) )
+ {
+ beginTime = true;
+ }
+ if (difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())
+ && difference.getControlNodeDetail().getXpathLocation().endsWith("@end")
+ && (Integer.parseInt(difference.getControlNodeDetail().getValue()) > Integer.parseInt(difference.getTestNodeDetail().getValue())) )
+ {
+ endTime = true;
+ }
+ //attribute values of @occurrence and @corpus are ignored if they are different
+ if (difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())
+ && difference.getControlNodeDetail().getXpathLocation().endsWith("@occurrence"))
+ {
+ occDiff = true;
+ }
+ if (difference.getControlNodeDetail().getXpathLocation().equals(difference.getTestNodeDetail().getXpathLocation())
+ && difference.getControlNodeDetail().getXpathLocation().endsWith("@corpus"))
+ {
+ corpusDiff = true;
+ }
+ }
+ //if any of @begin, @end ... is identical set Boolean to true
+ if (i == 0) beginTime = true;
+ if (j == 0) endTime = true;
+ if (k == 0) occDiff = true;
+ if (l == 0) corpusDiff = true;
+ //if the size of differences is greater than the number of changes admitted in @begin, @end ... something else must be different
+ if (beginTime && endTime && occDiff && corpusDiff && (i + j + k + l) == dd.getAllDifferences().size()) isEqual = true;
+ }
+ catch (SAXException e)
+ {
+ e.printStackTrace();
+ }
+ catch (IOException e)
+ {
+ e.printStackTrace();
+ }
+ return isEqual;
}
- return isEqual;
-}
-\end{lstlisting}
+
In this method, XMLUnit is heavily used to make all necessary node comparisons. The matter becomes more complicated, however, if some attributes
are not only ignored, but evaluated according to a given definition as it is the case for the time range. If the evaluator and builder classes are
not to be overwritten entirely because needed for evaluating other nodes of the
xml document, the above solution appears a bit awkward. So there is potential for improvement before the production version is to be programmed.
XMLUnit provides us with a
-list of the differences of the two documents (see line \ref{ln:difflist}). There are four differences allowed, that is, the attributes \emph{occurrence},
-\emph{corpus}, \emph{begin}, and \emph{end}. For each of them a Boolean variable is set. Because any of the attributes could also be equal to the master
+list of the differences of the two documents (see line 29). There are four differences allowed, that is, the attributes *occurrence*,
+*corpus*, *begin*, and *end*. For each of them a Boolean variable is set. Because any of the attributes could also be equal to the master
document and the difference list only contains the actual differences, one has to find a way to define both, equal and different, for the attributes.
This could be done by ignoring these nodes. Yet, this would not include testing if the beginning and ending dates fall into the range of the master
-document. Therefore the attributes are counted as lines \ref{ln:diffbegin} through \ref{ln:diffend} reveal. If any two documents
-differ in some of the four attributes just specified, then the sum of the counters (line \ref{ln:diffsum}) should not be greater than the collected differences
+document. Therefore the attributes are counted as lines 34 through 37 reveal. If any two documents
+differ in some of the four attributes just specified, then the sum of the counters (line 69) should not be greater than the collected differences
by XMLUnit. The rest of the if-tests assign truth values to the respective
Booleans. It is probably worth mentioning that if all counters are zero (lines
-\ref{ln:zerobegin}-\ref{ln:zeroend}) the attributes and values are identical and hence the Boolean has to be set explicitly. Otherwise the test in line \ref{ln:diffsum} would fail.
+64--67) the attributes and values are identical and hence the Boolean has to be set explicitly. Otherwise the test in line 69 would fail.
-%TagCorpusServlet
Once quality control (explained in detail further down) has been passed, it is
-the user's turn to interact further. By clicking on the option \emph{Manual tagging}, the \emph{TagCorpusServlet} will be callled. This servlet instantiates
-\emph{ProcessCorpusServlet} to get access to the \emph{getUnknownWords}-method, which delivers the words still to be
-processed and which overwrites the content of the file starting with \emph{untagged}. For the next word in \emph{leftovers} a new MyCoRe object is created
-using the JDOM API and added to the file beginning with \emph{processed}. In line \ref{ln:tagmanu} of listing \ref{src:tagservlet}, the previously defined
+the user's turn to interact further. By clicking on the option *Manual tagging*, the *TagCorpusServlet* will be callled. This servlet instantiates
+*ProcessCorpusServlet* to get access to the *getUnknownWords*-method, which delivers the words still to be
+processed and which overwrites the content of the file starting with *untagged*. For the next word in *leftovers* a new MyCoRe object is created
+using the JDOM API and added to the file beginning with *processed*. In line 16 of listing :ref:`tagservlet`, the previously defined
entry mask is called, with which the proposed word structure could be confirmed or changed. How the word structure is determined will be shown later in
the text.
-\begin{lstlisting}[language=java,caption={Manual Tagging Procedure},label=src:tagservlet,escapechar=|]
-...
-if (!leftovers.isEmpty())
-{
- ArrayList<String> processed = new ArrayList<String>();
- //processed.add(leftovers.get(0));
- JDOMorphilo jdm = new JDOMorphilo();
- MCRObject obj = jdm.createMorphiloObject(job, leftovers.get(0));|\label{ln:jdomobject}|
- //write word to be annotated in process list and save it
- Path filePathProc = pcs.getDerivateFilePath(job, "processed").getFileName();
- Path proc = root.resolve(filePathProc);
- processed = pcs.getContentFromFile(job, "processed");
- processed.add(leftovers.get(0));
- Files.write(proc, processed);
-
- //call entry mask for next word
- tagUrl = prop.getBaseURL() + "content/publish/morphilo.xed?id=" + obj.getId();|\label{ln:tagmanu}|
-}
-else
-{
- //initiate process to give a complete tagged file of the original corpus
- //if untagged-file is empty, match original file with morphilo
- //creator=administrator OR creator=username and write matches in a new file
- ArrayList<String> complete = new ArrayList<String>();
- ProcessCorpusServlet pcs2 = new ProcessCorpusServlet();
- complete = pcs2.getUnknownWords(
- pcs2.getContentFromFile(job, ""), //main corpus file
- pcs2.getCorpusMetadata(job, "def.datefrom"),
- pcs2.getCorpusMetadata(job, "def.dateuntil"),
- "", //wordtype
- false,
- false,
- true);
-
- Files.delete(p);
- MCRXMLFunctions mdm = new MCRXMLFunctions();
- String mainFile = mdm.getMainDocName(derivID);
- Path newRoot = root.resolve("tagged-" + mainFile);
- Files.write(newRoot, complete);
-
- //return to Menu page
- tagUrl = prop.getBaseURL() + "receive/" + corpID;
-}
-\end{lstlisting}
-At the point where no more items are in \emph{leftsovers} the \emph{getUnknownWords}-method is called whereas the last Boolean parameter
+
+.. _tagservlet:
+
+.. code-block:: java
+ :caption: Manual Tagging Procedure
+ :linenos:
+
+ ...
+ if (!leftovers.isEmpty())
+ {
+ ArrayList<String> processed = new ArrayList<String>();
+ //processed.add(leftovers.get(0));
+ JDOMorphilo jdm = new JDOMorphilo();
+ MCRObject obj = jdm.createMorphiloObject(job, leftovers.get(0));
+ //write word to be annotated in process list and save it
+ Path filePathProc = pcs.getDerivateFilePath(job, "processed").getFileName();
+ Path proc = root.resolve(filePathProc);
+ processed = pcs.getContentFromFile(job, "processed");
+ processed.add(leftovers.get(0));
+ Files.write(proc, processed);
+
+ //call entry mask for next word
+ tagUrl = prop.getBaseURL() + "content/publish/morphilo.xed?id=" + obj.getId();
+ }
+ else
+ {
+ //initiate process to give a complete tagged file of the original corpus
+ //if untagged-file is empty, match original file with morphilo
+ //creator=administrator OR creator=username and write matches in a new file
+ ArrayList<String> complete = new ArrayList<String>();
+ ProcessCorpusServlet pcs2 = new ProcessCorpusServlet();
+ complete = pcs2.getUnknownWords(
+ pcs2.getContentFromFile(job, ""), //main corpus file
+ pcs2.getCorpusMetadata(job, "def.datefrom"),
+ pcs2.getCorpusMetadata(job, "def.dateuntil"),
+ "", //wordtype
+ false,
+ false,
+ true);
+
+ Files.delete(p);
+ MCRXMLFunctions mdm = new MCRXMLFunctions();
+ String mainFile = mdm.getMainDocName(derivID);
+ Path newRoot = root.resolve("tagged-" + mainFile);
+ Files.write(newRoot, complete);
+
+ //return to Menu page
+ tagUrl = prop.getBaseURL() + "receive/" + corpID;
+ }
+
+At the point where no more items are in *leftsovers* the *getUnknownWords*-method is called whereas the last Boolean parameter
is set true. This indicates that the array list containing all available and relevant data to the respective user is returned as seen in
the code snippet in listing \ref{src:writeAll}.
-\begin{lstlisting}[language=java,caption={Code snippet to deliver all data to the user},label=src:writeAll,escapechar=|]
-...
-// all data is written to lo in TEI
-if (writeAllData && isAuthorized && timeCorrect)
-{
- XPathExpression<Element> xpath = xpfac.compile("//morphiloContainer/morphilo", Filters.element());
- for (Element e : xpath.evaluate(jdomDoc))
- {
- XMLOutputter outputter = new XMLOutputter();
- outputter.setFormat(Format.getPrettyFormat());
- lo.add(outputter.outputString(e.getContent()));
- }
-}
-...
-\end{lstlisting}
-The complete list (\emph{lo}) is written to yet a third file starting with \emph{tagged} and finally returned to the main project webpage.
-
-%JDOMorphilo
+
+.. code-block:: java
+ :caption: Code snippet to deliver all data to the user
+
+ ...
+ // all data is written to lo in TEI
+ if (writeAllData && isAuthorized && timeCorrect)
+ {
+ XPathExpression<Element> xpath = xpfac.compile("//morphiloContainer/morphilo", Filters.element());
+ for (Element e : xpath.evaluate(jdomDoc))
+ {
+ XMLOutputter outputter = new XMLOutputter();
+ outputter.setFormat(Format.getPrettyFormat());
+ lo.add(outputter.outputString(e.getContent()));
+ }
+ }
+ ...
+
+The complete list (*lo*) is written to yet a third file starting with *tagged* and finally returned to the main project webpage.
+
The interesting question is now where does the word structure come from, which is filled in the entry mask as asserted above.
-In listing \ref{src:tagservlet} line \ref{ln:jdomobject}, one can see that a JDOM object is created and the method
-\emph{createMorphiloObject(MCRServletJob, String)} is called. The string parameter is the word that needs to be analyzed.
-Most of the method is a mere application of the JDOM API given the data model in chapter \ref{chap:concept} section
-\ref{subsec:datamodel} and listing \ref{lst:worddatamodel}. That means namespaces, elements and their attributes are defined in the correct
+In listing :ref:`tagservlet` line 7, one can see that a JDOM object is created and the method
+*createMorphiloObject(MCRServletJob, String)* is called. The string parameter is the word that needs to be analyzed.
+Most of the method is a mere application of the JDOM API given the data model in :ref:`concept` and listing :ref:`worddatamodel`. That means namespaces, elements and their attributes are defined in the correct
order and hierarchy.
To fill the elements and attributes with text, i.e. prefixes, suffixes, stems, etc., a Hashmap -- containing the morpheme as
@@ -594,256 +631,292 @@ key and its position as value -- are created that are filled with the results fr
or suffixes respectively are put in the hashmap, the same number of xml elements are created. As a final step, a valid MyCoRe id is generated using
the existing MyCoRe functionality, the object is created and returned to the TagCorpusServlet.
-%AffixStripper explanation
Last, the analyses of the word structure will be considered. It is implemented
-in the \emph{AffixStripper.java} file.
+in the *AffixStripper.java* file.
All lexical affix morphemes and their allomorphs as well as the inflections were extracted from the
-OED\footnote{Oxford English Dictionary http://www.oed.com/} and saved as enumerated lists (see the example in listing \ref{src:enumPref}).
+`Oxford English Dictionary <http://www.oed.com/>`_ and saved as enumerated lists (see the example in listing :ref:`enumPref`).
The allomorphic items of these lists are mapped successively to the beginning in the case of prefixes
-(see listing \ref{src:analyzePref}, line \ref{ln:prefLoop}) or to the end of words in the case of suffixes
-(see listing \ref{src:analyzeSuf}). Since each
+(see listing :ref:`analyzePref`, line 7) or to the end of words in the case of suffixes
+(see listing :ref:`analyzeSuf`). Since each
morphemic variant maps to its morpheme right away, it makes sense to use the morpheme and so
implicitly keep the relation to its allomorph.
-\begin{lstlisting}[language=java,caption={Enumeration Example for the Prefix "over"},label=src:enumPref,escapechar=|]
-package custom.mycore.addons.morphilo;
-
-public enum PrefixEnum {
-...
- over("over"), ufer("over"), ufor("over"), uferr("over"), uvver("over"), obaer("over"), ober("over)"), ofaer("over"),
- ofere("over"), ofir("over"), ofor("over"), ofer("over"), ouer("over"),oferr("over"), offerr("over"), offr("over"), aure("over"),
- war("over"), euer("over"), oferre("over"), oouer("over"), oger("over"), ouere("over"), ouir("over"), ouire("over"),
- ouur("over"), ouver("over"), ouyr("over"), ovar("over"), overe("over"), ovre("over"),ovur("over"), owuere("over"), owver("over"),
- houyr("over"), ouyre("over"), ovir("over"), ovyr("over"), hover("over"), auver("over"), awver("over"), ovver("over"),
- hauver("over"), ova("over"), ove("over"), obuh("over"), ovah("over"), ovuh("over"), ofowr("over"), ouuer("over"), oure("over"),
- owere("over"), owr("over"), owre("over"), owur("over"), owyr("over"), our("over"), ower("over"), oher("over"),
- ooer("over"), oor("over"), owwer("over"), ovr("over"), owir("over"), oar("over"), aur("over"), oer("over"), ufara("over"),
- ufera("over"), ufere("over"), uferra("over"), ufora("over"), ufore("over"), ufra("over"), ufre("over"), ufyrra("over"),
- yfera("over"), yfere("over"), yferra("over"), uuera("over"), ufe("over"), uferre("over"), uuer("over"), uuere("over"),
- vfere("over"), vuer("over"), vuere("over"), vver("over"), uvvor("over") ...
-...chap:results
- private String morpheme;
- //constructor
- PrefixEnum(String morpheme)
- {
- this.morpheme = morpheme;
- }
- //getter Method
-
- public String getMorpheme()
- {
- return this.morpheme;
- }
-}
-\end{lstlisting}
-As can be seen in line \ref{ln:prefPutMorph} in listing \ref{src:analyzePref}, the morpheme is saved to a hash map together with its position, i.e. the size of the
-map plus one at the time being. In line \ref{ln:prefCutoff} the \emph{analyzePrefix} method is recursively called until no more matches can be made.
-
-\begin{lstlisting}[language=java,caption={Method to recognize prefixes},label=src:analyzePref,escapechar=|]
-private Map<String, Integer> prefixMorpheme = new HashMap<String,Integer>();
-...
-private void analyzePrefix(String restword)
-{
- if (!restword.isEmpty()) //Abbruchbedingung fuer Rekursion
- {
- for (PrefixEnum prefEnum : PrefixEnum.values())|\label{ln:prefLoop}|
- {
- String s = prefEnum.toString();
- if (restword.startsWith(s))
+.. _enumPref:
+
+.. code-block:: java
+ :caption: Enumeration Example for the Prefix *over*
+
+ package custom.mycore.addons.morphilo;
+ public enum PrefixEnum {
+ ...
+ over("over"), ufer("over"), ufor("over"), uferr("over"), uvver("over"), obaer("over"), ober("over)"), ofaer("over"),
+ ofere("over"), ofir("over"), ofor("over"), ofer("over"), ouer("over"),oferr("over"), offerr("over"), offr("over"), aure("over"),
+ war("over"), euer("over"), oferre("over"), oouer("over"), oger("over"), ouere("over"), ouir("over"), ouire("over"),
+ ouur("over"), ouver("over"), ouyr("over"), ovar("over"), overe("over"), ovre("over"),ovur("over"), owuere("over"), owver("over"),
+ houyr("over"), ouyre("over"), ovir("over"), ovyr("over"), hover("over"), auver("over"), awver("over"), ovver("over"),
+ hauver("over"), ova("over"), ove("over"), obuh("over"), ovah("over"), ovuh("over"), ofowr("over"), ouuer("over"), oure("over"),
+ owere("over"), owr("over"), owre("over"), owur("over"), owyr("over"), our("over"), ower("over"), oher("over"),
+ ooer("over"), oor("over"), owwer("over"), ovr("over"), owir("over"), oar("over"), aur("over"), oer("over"), ufara("over"),
+ ufera("over"), ufere("over"), uferra("over"), ufora("over"), ufore("over"), ufra("over"), ufre("over"), ufyrra("over"),
+ yfera("over"), yfere("over"), yferra("over"), uuera("over"), ufe("over"), uferre("over"), uuer("over"), uuere("over"),
+ vfere("over"), vuer("over"), vuere("over"), vver("over"), uvvor("over") ...
+ private String morpheme;
+ //constructor
+ PrefixEnum(String morpheme)
{
- prefixMorpheme.put(s, prefixMorpheme.size() + 1);|\label{ln:prefPutMorph}|
- //cut off the prefix that is added to the list
- analyzePrefix(restword.substring(s.length()));|\label{ln:prefCutoff}|
+ this.morpheme = morpheme;
}
- else
+ //getter Method
+
+ public String getMorpheme()
{
- analyzePrefix("");
+ return this.morpheme;
+ }
+ }
+
+As can be seen in line 12 in listing :ref:`analyzePref`, the morpheme is saved to a hash map together with its position, i.e. the size of the
+map plus one at the time being. In line 14 the *analyzePrefix* method is recursively called until no more matches can be made.
+
+.. _analyzePref:
+
+.. code-block:: java
+ :caption: Method to recognize prefixes
+ :linenos:
+ :emphasize-lines: 12
+
+ private Map<String, Integer> prefixMorpheme = new HashMap<String,Integer>();
+ ...
+ private void analyzePrefix(String restword)
+ {
+ if (!restword.isEmpty()) //Abbruchbedingung fuer Rekursion
+ {
+ for (PrefixEnum prefEnum : PrefixEnum.values())
+ {
+ String s = prefEnum.toString();
+ if (restword.startsWith(s))
+ {
+ prefixMorpheme.put(s, prefixMorpheme.size() + 1);
+ //cut off the prefix that is added to the list
+ analyzePrefix(restword.substring(s.length()));
+ }
+ else
+ {
+ analyzePrefix("");
+ }
+ }
}
}
- }
-}
-\end{lstlisting}
The recognition of suffixes differs only in the cut-off direction since suffixes occur at the end of a word.
-Hence, line \ref{ln:prefCutoff} in listing \ref{src:analyzePref} reads in the case of suffixes.
+Hence, line 14 in listing :ref:`analyzePref` reads in the case of suffixes.
-\begin{lstlisting}[language=java,caption={Cut-off mechanism for suffixes},label=src:analyzeSuf,escapechar=|]
-analyzeSuffix(restword.substring(0, restword.length() - s.length()));
-\end{lstlisting}
+.. _analyzeSuf:
+
+.. code-block:: java
+ :caption: Cut-off mechanism for suffixes
+
+ analyzeSuffix(restword.substring(0, restword.length() - s.length()));
It is important to note that inflections are suffixes (in the given model case of Middle English morphology) that usually occur at the very
end of a word, i.e. after all lexical suffixes, only once. It follows that inflections
have to be recognized at first without any repetition. So the procedure for inflections can be simplified
-to a substantial degree as listing \ref{src:analyzeInfl} shows.
-
-\begin{lstlisting}[language=java,caption={Method to recognize inflections},label=src:analyzeInfl,escapechar=|]
-private String analyzeInflection(String wrd)
-{
- String infl = "";
- for (InflectionEnum inflEnum : InflectionEnum.values())
- {
- if (wrd.endsWith(inflEnum.toString()))
+to a substantial degree as listing :ref:`analyzeInfl` shows.
+
+.. _analyzeInfl:
+
+.. code-block:: java
+ :caption: Method to recognize inflections
+
+ private String analyzeInflection(String wrd)
{
- infl = inflEnum.toString();
+ String infl = "";
+ for (InflectionEnum inflEnum : InflectionEnum.values())
+ {
+ if (wrd.endsWith(inflEnum.toString()))
+ {
+ infl = inflEnum.toString();
+ }
+ }
+ return infl;
}
- }
- return infl;
-}
-\end{lstlisting}
Unfortunately the embeddedness problem prevents a very simple algorithm. Embeddedness occurs when a lexical item
-is a substring of another lexical item. To illustrate, the suffix \emph{ion} is also contained in the suffix \emph{ation}, as is
-\emph{ent} in \emph{ment}, and so on. The embeddedness problem cannot be solved completely on the basis of linear modelling, but
+is a substring of another lexical item. To illustrate, the suffix *ion* is also contained in the suffix *ation*, as is
+*ent* in *ment*, and so on. The embeddedness problem cannot be solved completely on the basis of linear modelling, but
for a large part of embedded items one can work around it using implicitly Zipf's law, i.e. the correlation between frequency
and length of lexical items. The longer a word becomes, the less frequent it will occur. The simplest logic out of it is to assume
that longer suffixes (measured in letters) are preferred over shorter suffixes because it is more likely tha the longer the suffix string becomes,
-the more likely it is one (as opposed to several) suffix unit(s). This is done in listing \ref{src:embedAffix}, whereas
-the inner class \emph{sortedByLengthMap} returns a list sorted by length and the loop from line \ref{ln:deleteAffix} onwards deletes
+the more likely it is one (as opposed to several) suffix unit(s). This is done in listing :ref:`embedAffix`, whereas
+the inner class *sortedByLengthMap* returns a list sorted by length and the loop from line 17 onwards deletes
the respective substrings.
-\begin{lstlisting}[language=java,caption={Method to workaround embeddedness},label=src:embedAffix,escapechar=|]
-private Map<String, Integer> sortOutAffixes(Map<String, Integer> affix)
-{
- Map<String,Integer> sortedByLengthMap = new TreeMap<String, Integer>(new Comparator<String>()
+.. _embedAffix:
+
+.. code-block:: java
+ :caption: Method to workaround embeddedness
+ :linenos:
+
+ private Map<String, Integer> sortOutAffixes(Map<String, Integer> affix)
{
- @Override
- public int compare(String s1, String s2)
- {
- int cmp = Integer.compare(s1.length(), s2.length());
- return cmp != 0 ? cmp : s1.compareTo(s2);
- }
- }
- );
- sortedByLengthMap.putAll(affix);
- ArrayList<String> al1 = new ArrayList<String>(sortedByLengthMap.keySet());
- ArrayList<String> al2 = al1;
- Collections.reverse(al2);
- for (String s2 : al1)|\label{ln:deleteAffix}|
- {
- for (String s1 : al2)
- if (s1.contains(s2) && s1.length() > s2.length())
+ Map<String,Integer> sortedByLengthMap = new TreeMap<String, Integer>(new Comparator<String>()
+ {
+ @Override
+ public int compare(String s1, String s2)
+ {
+ int cmp = Integer.compare(s1.length(), s2.length());
+ return cmp != 0 ? cmp : s1.compareTo(s2);
+ }
+ }
+ );
+ sortedByLengthMap.putAll(affix);
+ ArrayList<String> al1 = new ArrayList<String>(sortedByLengthMap.keySet());
+ ArrayList<String> al2 = al1;
+ Collections.reverse(al2);
+ for (String s2 : al1)
{
- affix.remove(s2);
- }
+ for (String s1 : al2)
+ if (s1.contains(s2) && s1.length() > s2.length())
+ {
+ affix.remove(s2);
+ }
+ }
+ return affix;
}
- return affix;
-}
-\end{lstlisting}
-
-Finally, the position of the affix has to be calculated because the hashmap in line \ref{ln:prefPutMorph} in
-listing \ref{src:analyzePref} does not keep the original order for changes taken place in addressing the affix embeddedness
-(listing \ref{src:embedAffix}). Listing \ref{src:affixPos} depicts the preferred solution.
-The recursive construction of the method is similar to \emph{private void analyzePrefix(String)} (listing \ref{src:analyzePref})
-only that the two affix types are handled in one method. For that, an additional parameter taking the form either \emph{suffix}
-or \emph{prefix} is included.
-
-\begin{lstlisting}[language=java,caption={Method to determine position of the affix},label=src:affixPos,escapechar=|]
-private void getAffixPosition(Map<String, Integer> affix, String restword, int pos, String affixtype)
-{
- if (!restword.isEmpty()) //Abbruchbedingung fuer Rekursion
- {
- for (String s : affix.keySet())
+
+Finally, the position of the affix has to be calculated because the hashmap in line 12 in
+listing :ref:`analyzePref` does not keep the original order for changes taken place in addressing the affix embeddedness
+(listing :ref:`embedAffix`). Listing :ref:`affixPos` depicts the preferred solution.
+The recursive construction of the method is similar to *private void analyzePrefix(String)* (listing :ref:`analyzePref`)
+only that the two affix types are handled in one method. For that, an additional parameter taking the form either *suffix*
+or *prefix* is included.
+
+.. _affixPos:
+
+.. code-block:: java
+ :caption: Method to determine position of the affix
+
+ private void getAffixPosition(Map<String, Integer> affix, String restword, int pos, String affixtype)
{
- if (restword.startsWith(s) && affixtype.equals("prefix"))
- {
- pos++;
- prefixMorpheme.put(s, pos);
- //prefixAllomorph.add(pos-1, restword.substring(s.length()));
- getAffixPosition(affix, restword.substring(s.length()), pos, affixtype);
- }
- else if (restword.endsWith(s) && affixtype.equals("suffix"))
- {
- pos++;
- suffixMorpheme.put(s, pos);
- //suffixAllomorph.add(pos-1, restword.substring(s.length()));
- getAffixPosition(affix, restword.substring(0, restword.length() - s.length()), pos, affixtype);
- }
- else
+ if (!restword.isEmpty()) //Abbruchbedingung fuer Rekursion
{
- getAffixPosition(affix, "", pos, affixtype);
+ for (String s : affix.keySet())
+ {
+ if (restword.startsWith(s) && affixtype.equals("prefix"))
+ {
+ pos++;
+ prefixMorpheme.put(s, pos);
+ //prefixAllomorph.add(pos-1, restword.substring(s.length()));
+ getAffixPosition(affix, restword.substring(s.length()), pos, affixtype);
+ }
+ else if (restword.endsWith(s) && affixtype.equals("suffix"))
+ {
+ pos++;
+ suffixMorpheme.put(s, pos);
+ //suffixAllomorph.add(pos-1, restword.substring(s.length()));
+ getAffixPosition(affix, restword.substring(0, restword.length() - s.length()), pos, affixtype);
+ }
+ else
+ {
+ getAffixPosition(affix, "", pos, affixtype);
+ }
+ }
}
}
- }
-}
-\end{lstlisting}
-To give the complete word structure, the root of a word should also be provided. In listing \ref{src:rootAnalyze} a simple solution is offered, however,
+To give the complete word structure, the root of a word should also be provided. In listing :ref:`rootAnalyze` a simple solution is offered, however,
considering compounds as words consisting of more than one root.
-\begin{lstlisting}[language=java,caption={Method to determine roots},label=src:rootAnalyze,escapechar=|]
-private ArrayList<String> analyzeRoot(Map<String, Integer> pref, Map<String, Integer> suf, int stemNumber)
-{
- ArrayList<String> root = new ArrayList<String>();
- int j = 1; //one root always exists
- // if word is a compound several roots exist
- while (j <= stemNumber)
- {
- j++;
- String rest = lemma;|\label{ln:lemma}|
-
- for (int i=0;i<pref.size();i++)
+
+.. _rootAnalyze:
+
+.. code-block:: java
+ :caption: Method to determine roots
+
+ private ArrayList<String> analyzeRoot(Map<String, Integer> pref, Map<String, Integer> suf, int stemNumber)
{
- for (String s : pref.keySet())
+ ArrayList<String> root = new ArrayList<String>();
+ int j = 1; //one root always exists
+ // if word is a compound several roots exist
+ while (j <= stemNumber)
{
- //if (i == pref.get(s))
- if (rest.length() > s.length() && s.equals(rest.substring(0, s.length())))
- {
- rest = rest.substring(s.length(),rest.length());
+ j++;
+ String rest = lemma;
+
+ for (int i=0;i<pref.size();i++)
+ {
+ for (String s : pref.keySet())
+ {
+ //if (i == pref.get(s))
+ if (rest.length() > s.length() && s.equals(rest.substring(0, s.length())))
+ {
+ rest = rest.substring(s.length(),rest.length());
+ }
+ }
}
+
+ for (int i=0;i<suf.size();i++)
+ {
+ for (String s : suf.keySet())
+ {
+ //if (i == suf.get(s))
+ if (s.length() < rest.length() && (s.equals(rest.substring(rest.length() - s.length(), rest.length()))))
+ {
+ rest = rest.substring(0, rest.length() - s.length());
+ }
+ }
+ }
+ root.add(rest);
}
+ return root;
}
-
- for (int i=0;i<suf.size();i++)
- {
- for (String s : suf.keySet())
- {
- //if (i == suf.get(s))
- if (s.length() < rest.length() && (s.equals(rest.substring(rest.length() - s.length(), rest.length()))))
- {
- rest = rest.substring(0, rest.length() - s.length());
- }
- }
- }
- root.add(rest);
- }
- return root;
-}
-\end{lstlisting}
+
The logic behind this method is that the root is the remainder of a word when all prefixes and suffixes are substracted.
So the loops run through the number of prefixes and suffixes at each position and substract the affix. Really, there is
some code doubling with the previously described methods, which could be eliminated by making it more modular in a possible
-refactoring phase. Again, this is not the concern of a prototype. Line \ref{ln:lemma} defines the initial state of a root,
-which is the case for monomorphemic words. The \emph{lemma} is defined as the wordtoken without the inflection. Thus listing
-\ref{src:lemmaAnalyze} reveals how the class variable is calculated
-\begin{lstlisting}[language=java,caption={Method to determine lemma},label=src:lemmaAnalyze,escapechar=|]
-/*
- * Simplification: lemma = wordtoken - inflection
- */
-private String analyzeLemma(String wrd, String infl)
-{
- return wrd.substring(0, wrd.length() - infl.length());
-}
-\end{lstlisting}
-The constructor of \emph{AffixStripper} calls the method \emph{analyzeWord()}
+refactoring phase. Again, this is not the concern of a prototype. Line 9 defines the initial state of a root,
+which is the case for monomorphemic words. The *lemma* is defined as the wordtoken without the inflection. Thus listing
+:ref:`lemmaAnalyze` reveals how the class variable is calculated
+
+.. _lemmaAnalyze:
+
+.. code-block:: java
+ :caption: Method to determine lemma
+
+ /*
+ * Simplification: lemma = wordtoken - inflection
+ */
+ private String analyzeLemma(String wrd, String infl)
+ {
+ return wrd.substring(0, wrd.length() - infl.length());
+ }
+
+The constructor of *AffixStripper* calls the method *analyzeWord()*
whose only job is to calculate each structure element in the correct order
-(listing \ref{src:lemmaAnalyze}). All structure elements are also provided by getters.
-\begin{lstlisting}[language=java,caption={Method to determine all word structure},label=src:lemmaAnalyze,escapechar=|]
-private void analyzeWord()
-{
- //analyze inflection first because it always occurs at the end of a word
- inflection = analyzeInflection(wordtoken);
- lemma = analyzeLemma(wordtoken, inflection);
- analyzePrefix(lemma);
- analyzeSuffix(lemma);
- getAffixPosition(sortOutAffixes(prefixMorpheme), lemma, 0, "prefix");
- getAffixPosition(sortOutAffixes(suffixMorpheme), lemma, 0, "suffix");
- prefixNumber = prefixMorpheme.size();
- suffixNumber = suffixMorpheme.size();
- wordroot = analyzeRoot(prefixMorpheme, suffixMorpheme, getStemNumber());
-}
-\end{lstlisting}
+(listing :ref:`lemmaAnalyze`. All structure elements are also provided by getters.
+
+
+.. code-block:: java
+ :caption: Method to determine all word structure
+
+ private void analyzeWord()
+ {
+ //analyze inflection first because it always occurs at the end of a word
+ inflection = analyzeInflection(wordtoken);
+ lemma = analyzeLemma(wordtoken, inflection);
+ analyzePrefix(lemma);
+ analyzeSuffix(lemma);
+ getAffixPosition(sortOutAffixes(prefixMorpheme), lemma, 0, "prefix");
+ getAffixPosition(sortOutAffixes(suffixMorpheme), lemma, 0, "suffix");
+ prefixNumber = prefixMorpheme.size();
+ suffixNumber = suffixMorpheme.size();
+ wordroot = analyzeRoot(prefixMorpheme, suffixMorpheme, getStemNumber());
+ }
To conclude, the Morphilo implementation as presented here, aims at fulfilling the task of a working prototype. It is important to note
that it neither claims to be a very efficient nor a ready software program to be used in production. However, it marks a crucial milestone
on the way to a production system. At some listings sources of improvement were made explicit; at others no suggestions were made. In the latter
case this does not imply that there is no potential for improvement. Once acceptability tests are carried out, it will be the task of a follow up project
-to identify these potentials and implement them accordingly.
\ No newline at end of file
+to identify these potentials and implement them accordingly.
+
+.. rubric:: Notes
+
+.. [#f4] This number is somewhat arbitrary. It is inspired by the sample size n in t-distributed data.
diff --git a/Morphilo_doc/source/datamodel.rst b/Morphilo_doc/source/datamodel.rst
index f206ef3ffb8967f600e499b5b76eee996ac7de31..6274276ba44db0cc5450b28bc208289b939610d5 100644
--- a/Morphilo_doc/source/datamodel.rst
+++ b/Morphilo_doc/source/datamodel.rst
@@ -1,6 +1,8 @@
Data Model
==========
+.. _concept:
+
Conceptualization
-----------------
@@ -11,24 +13,24 @@ and multi-user processing is necessary. In addition, the framework should
support web technologies, be well documented, and easy to extent. Ideally, the
MVC pattern is realized.
-\subsection{Data Model}\label{subsec:datamodel}
-The guidelines of the
-\emph{TEI}-standard\footnote{http://www.tei-c.org/release/doc/tei-p5-doc/en/Guidelines.pdf} on the
-word level are defined in line with the structure defined above in section \ref{subsec:morphologicalSystems}.
-In listing \ref{lst:teiExamp} an
+The guidelines of the `TEI standard <http://www.tei-c.org/release/doc/tei-p5-doc/en/Guidelines.pdf>`_ on the
+word level are defined in line with the defined word structure.
+In listing :ref:`teiexamp` an
example is given for a possible markup at the word level for
-\emph{comfortable}.\footnote{http://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-m.html}
-
-\begin{lstlisting}[language=XML,
-caption={TEI-example for 'comfortable'},label=lst:teiExamp]
-<w type="adjective">
- <m type="base">
- <m type="prefix" baseForm="con">com</m>
- <m type="root">fort</m>
- </m>
- <m type="suffix">able</m>
-</w>
-\end{lstlisting}
+`comfortable <http://www.tei-c.org/release/doc/tei-p5-doc/en/html/ref-m.html>`_
+
+.. _teiexamp:
+
+.. code-block:: xml
+ :caption: TEI-example for *comfortable*
+
+ <w type="adjective">
+ <m type="base">
+ <m type="prefix" baseForm="con">com</m>
+ <m type="root">fort</m>
+ </m>
+ <m type="suffix">able</m>
+ </w>
This data model reflects just one theoretical conception of a word structure model.
Crucially, the model emanates from the assumption
@@ -38,7 +40,7 @@ other hand, is enclosed in the base, which basically means a stronger lexical,
and less abstract, attachment to the root of a word. Modeling prefixes and suffixes on different
hierarchical levels has important consequences for the branching direction at
subword level (here right-branching). Left the theoretical interest aside, the
-choice of the TEI standard is reasonable with view to a sustainable architecture that allows for
+choice of the *TEI*-standard is reasonable with view to a sustainable architecture that allows for
exchanging data with little to no additional adjustments.
The negative account is that the model is not eligible for all languages.
@@ -51,191 +53,197 @@ stem and corresponds to the overwhelming majority of all research carried out
Implementation
--------------
-As laid out in the task analysis in section \ref{subsec:datamodel}, it is
-advantageous to use established standards. It was also shown that it makes sense
+It is
+advantageous to use established standardsn and it makes sense
to keep the meta data of each corpus separate from the data model used for the
words to be analyzed.
-For the present case, the TEI-standard was identified as an
+For the present case, the *TEI*-standard was identified as an
appropriate markup for words. In terms of the implementation this means that
-the TEI guidelines have to be implemented as an object type compatible with the chosen
+the *TEI*-guidelines have to be implemented as an object type compatible with the chosen
repository framework. However, the TEI standard is not complete regarding the
diachronic dimension, i.e. information on the development of the word. To
be compatible with the elements of the TEI standard on the one hand
and to best meet the requirements of the application on the other hand, some attributes
are added. This solution allows for processing the xml files according to
-the TEI standard by ignoring the additional attributes and at the same
+the *TEI*-standard by ignoring the additional attributes and at the same
time, if needed, additional markup can be extracted. The additional attributes
-comprise a link to the corpus meta data, but also \emph{position} and
-\emph{occurrence} of the affixes.
+comprise a link to the corpus meta data, but also *position* and
+*occurrence* of the affixes.
Information on the position and some quantification thereof are potentially relevant for a
wealth of research questions, such as predictions on the productivity of
derivatives and their interaction with the phonological or syntactic modules. So they were included
with respect to future use.
For reasons of efficiency in subsequent processing,
-the historic dates \emph{begin} and \emph{end} were included in both the word
+the historic dates *begin* and *end* were included in both the word
data model and the corpus data model. The result of the word data model is given
-in listing \ref{lst:worddatamodel}.
+in listing :ref:`worddatamodel`.
Whereas attributes of the objecttype are specific to the repository framework, the TEI structure can be
recognized in the hierarchy of the meta data element starting with the name
-\emph{w} (line \ref{src:wordbegin}).
-
-\begin{lstlisting}[language=XML,caption={Word Data
-model},label=lst:worddatamodel,escapechar=|] <?xml version="1.0" encoding="UTF-8"?>
-<objecttype
- name="morphilo"
- isChild="true"
- isParent="true"
- hasDerivates="true"
- xmlns:xs="http://www.w3.org/2001/XMLSchema"
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:noNamespaceSchemaLocation="datamodel.xsd">
- <metadata>
- <element name="morphiloContainer" type="xml" style="dontknow"
- notinherit="true" heritable="false">
- <xs:sequence>
- <xs:element name="morphilo">
- <xs:complexType>
- <xs:sequence>
- <xs:element name="w" minOccurs="0" maxOccurs="unbounded">|label{src:wordbegin}|
- <xs:complexType mixed="true">
- <xs:sequence>
- <!-- stem -->
- <xs:element name="m1" minOccurs="0" maxOccurs="unbounded">
- <xs:complexType mixed="true">
- <xs:sequence>
- <!-- base -->
- <xs:element name="m2" minOccurs="0" maxOccurs="unbounded">
- <xs:complexType mixed="true">
- <xs:sequence>
- <!-- root -->
- <xs:element name="m3" minOccurs="0" maxOccurs="unbounded">
- <xs:complexType mixed="true">
- <xs:attribute name="type" type="xs:string"/>
- </xs:complexType>
- </xs:element>
- <!-- prefix -->
- <xs:element name="m4" minOccurs="0" maxOccurs="unbounded">
- <xs:complexType mixed="true">
- <xs:attribute name="type" type="xs:string"/>
- <xs:attribute name="PrefixbaseForm" type="xs:string"/>
- <xs:attribute name="position" type="xs:string"/>
- </xs:complexType>
- </xs:element>
- </xs:sequence>
- <xs:attribute name="type" type="xs:string"/>
- </xs:complexType>
- </xs:element>
- <!-- suffix -->
- <xs:element name="m5" minOccurs="0" maxOccurs="unbounded">
- <xs:complexType mixed="true">
- <xs:attribute name="type" type="xs:string"/>
- <xs:attribute name="SuffixbaseForm" type="xs:string"/>
- <xs:attribute name="position" type="xs:string"/>
- <xs:attribute name="inflection" type="xs:string"/>
- </xs:complexType>
- </xs:element>
- </xs:sequence>
- <!-- stem-Attribute -->
- <xs:attribute name="type" type="xs:string"/>
- <xs:attribute name="pos" type="xs:string"/>
- <xs:attribute name="occurrence" type="xs:string"/>
- </xs:complexType>
- </xs:element>
- </xs:sequence>
- <!-- w -Attribute auf Wortebene -->
- <xs:attribute name="lemma" type="xs:string"/>
- <xs:attribute name="complexType" type="xs:string"/>
- <xs:attribute name="wordtype" type="xs:string"/>
- <xs:attribute name="occurrence" type="xs:string"/>
- <xs:attribute name="corpus" type="xs:string"/>
- <xs:attribute name="begin" type="xs:string"/>
- <xs:attribute name="end" type="xs:string"/>
- </xs:complexType>
- </xs:element>
- </xs:sequence>
- </xs:complexType>
- </xs:element>
- </xs:sequence>
- </element>
- <element name="wordtype" type="classification" minOccurs="0" maxOccurs="1">
- <classification id="wordtype"/>
- </element>
- <element name="complexType" type="classification" minOccurs="0" maxOccurs="1">
- <classification id="complexType"/>
- </element>
- <element name="corpus" type="classification" minOccurs="0" maxOccurs="1">
- <classification id="corpus"/>
- </element>
- <element name="pos" type="classification" minOccurs="0" maxOccurs="1">
- <classification id="pos"/>
- </element>
- <element name="PrefixbaseForm" type="classification" minOccurs="0"
- maxOccurs="1">
- <classification id="PrefixbaseForm"/>
- </element>
- <element name="SuffixbaseForm" type="classification" minOccurs="0"
- maxOccurs="1">
- <classification id="SuffixbaseForm"/>
- </element>
- <element name="inflection" type="classification" minOccurs="0" maxOccurs="1">
- <classification id="inflection"/>
- </element>
- <element name="corpuslink" type="link" minOccurs="0" maxOccurs="unbounded" >
- <target type="corpmeta"/>
- </element>
- </metadata>
-</objecttype>
-\end{lstlisting}
+*w* (line 17).
+
+.. _worddatamodel:
+
+.. code-block:: xml
+ :caption: Word Data Model
+ :linenos:
+ :emphasize-lines: 17
+
+ <?xml version="1.0" encoding="UTF-8"?>
+ <objecttype
+ name="morphilo"
+ isChild="true"
+ isParent="true"
+ hasDerivates="true"
+ xmlns:xs="http://www.w3.org/2001/XMLSchema"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:noNamespaceSchemaLocation="datamodel.xsd">
+ <metadata>
+ <element name="morphiloContainer" type="xml" style="dontknow"
+ notinherit="true" heritable="false">
+ <xs:sequence>
+ <xs:element name="morphilo">
+ <xs:complexType>
+ <xs:sequence>
+ <xs:element name="w" minOccurs="0" maxOccurs="unbounded">
+ <xs:complexType mixed="true">
+ <xs:sequence>
+ <!-- stem -->
+ <xs:element name="m1" minOccurs="0" maxOccurs="unbounded">
+ <xs:complexType mixed="true">
+ <xs:sequence>
+ <!-- base -->
+ <xs:element name="m2" minOccurs="0" maxOccurs="unbounded">
+ <xs:complexType mixed="true">
+ <xs:sequence>
+ <!-- root -->
+ <xs:element name="m3" minOccurs="0" maxOccurs="unbounded">
+ <xs:complexType mixed="true">
+ <xs:attribute name="type" type="xs:string"/>
+ </xs:complexType>
+ </xs:element>
+ <!-- prefix -->
+ <xs:element name="m4" minOccurs="0" maxOccurs="unbounded">
+ <xs:complexType mixed="true">
+ <xs:attribute name="type" type="xs:string"/>
+ <xs:attribute name="PrefixbaseForm" type="xs:string"/>
+ <xs:attribute name="position" type="xs:string"/>
+ </xs:complexType>
+ </xs:element>
+ </xs:sequence>
+ <xs:attribute name="type" type="xs:string"/>
+ </xs:complexType>
+ </xs:element>
+ <!-- suffix -->
+ <xs:element name="m5" minOccurs="0" maxOccurs="unbounded">
+ <xs:complexType mixed="true">
+ <xs:attribute name="type" type="xs:string"/>
+ <xs:attribute name="SuffixbaseForm" type="xs:string"/>
+ <xs:attribute name="position" type="xs:string"/>
+ <xs:attribute name="inflection" type="xs:string"/>
+ </xs:complexType>
+ </xs:element>
+ </xs:sequence>
+ <!-- stem-Attribute -->
+ <xs:attribute name="type" type="xs:string"/>
+ <xs:attribute name="pos" type="xs:string"/>
+ <xs:attribute name="occurrence" type="xs:string"/>
+ </xs:complexType>
+ </xs:element>
+ </xs:sequence>
+ <!-- w -Attribute auf Wortebene -->
+ <xs:attribute name="lemma" type="xs:string"/>
+ <xs:attribute name="complexType" type="xs:string"/>
+ <xs:attribute name="wordtype" type="xs:string"/>
+ <xs:attribute name="occurrence" type="xs:string"/>
+ <xs:attribute name="corpus" type="xs:string"/>
+ <xs:attribute name="begin" type="xs:string"/>
+ <xs:attribute name="end" type="xs:string"/>
+ </xs:complexType>
+ </xs:element>
+ </xs:sequence>
+ </xs:complexType>
+ </xs:element>
+ </xs:sequence>
+ </element>
+ <element name="wordtype" type="classification" minOccurs="0" maxOccurs="1">
+ <classification id="wordtype"/>
+ </element>
+ <element name="complexType" type="classification" minOccurs="0" maxOccurs="1">
+ <classification id="complexType"/>
+ </element>
+ <element name="corpus" type="classification" minOccurs="0" maxOccurs="1">
+ <classification id="corpus"/>
+ </element>
+ <element name="pos" type="classification" minOccurs="0" maxOccurs="1">
+ <classification id="pos"/>
+ </element>
+ <element name="PrefixbaseForm" type="classification" minOccurs="0"
+ maxOccurs="1">
+ <classification id="PrefixbaseForm"/>
+ </element>
+ <element name="SuffixbaseForm" type="classification" minOccurs="0"
+ maxOccurs="1">
+ <classification id="SuffixbaseForm"/>
+ </element>
+ <element name="inflection" type="classification" minOccurs="0" maxOccurs="1">
+ <classification id="inflection"/>
+ </element>
+ <element name="corpuslink" type="link" minOccurs="0" maxOccurs="unbounded" >
+ <target type="corpmeta"/>
+ </element>
+ </metadata>
+ </objecttype>
Additionally, it is worth mentioning that some attributes are modeled as a
-\emph{classification}. All these have to be listed
+*classification*. All these have to be listed
as separate elements in the data model. This has been done for all attributes
that are more or less subject to little or no change. In fact, all known suffix
and prefix morphemes should be known for the language investigated and are
therefore defined as a classification.
-The same is true for the parts of speech named \emph{pos} in the morphilo data
+The same is true for the parts of speech named *pos* in the morphilo data
model above.
Here the PENN-Treebank tagset was used. Last, the different morphemic layers in
-the standard model named \emph{m} are changed to $m1$ through $m5$. This is the
+the standard model named *m* are changed to *m1* through *m5*. This is the
only change in the standard that could be problematic if the data is to be
processed elsewhere and the change is not documented more explicitly. Yet, this
change was necessary for the MyCoRe repository throws errors caused by ambiguity
-issues on the different $m$-layers.
+issues on the different *m*-layers.
The second data model describes only very few properties of the text corpora
-from which the words are extracted. Listing \ref{lst:corpusdatamodel} depicts
+from which the words are extracted. Listing :ref:`corpusdatamodel` depicts
only the meta data element. For the sake of simplicity of the prototype, this
data model is kept as simple as possible. The obligatory field is the name of
the corpus. Specific dates of the corpus are classified as optional because in
some cases a text cannot be dated reliably.
-
-\begin{lstlisting}[language=XML,caption={Corpus Data
-Model},label=lst:corpusdatamodel]
-<metadata>
- <!-- Pflichtfelder -->
- <element name="korpusname" type="text" minOccurs="1" maxOccurs="1"/>
- <!-- Optionale Felder -->
- <element name="sprache" type="text" minOccurs="0" maxOccurs="1"/>
- <element name="size" type="number" minOccurs="0" maxOccurs="1"/>
- <element name="datefrom" type="text" minOccurs="0" maxOccurs="1"/>
- <element name="dateuntil" type="text" minOccurs="0" maxOccurs="1"/>
- <!-- number of words -->
- <element name="NoW" type="text" minOccurs="0" maxOccurs="1"/>
- <element name="corpuslink" type="link" minOccurs="0" maxOccurs="unbounded">
- <target type="morphilo"/>
- </element>
-</metadata>
-\end{lstlisting}
+.. _corpusdatamodel:
+
+.. code-block:: xml
+ :caption: Corpus Data Model
+
+ <metadata>
+ <!-- Pflichtfelder -->
+ <element name="korpusname" type="text" minOccurs="1" maxOccurs="1"/>
+ <!-- Optionale Felder -->
+ <element name="sprache" type="text" minOccurs="0" maxOccurs="1"/>
+ <element name="size" type="number" minOccurs="0" maxOccurs="1"/>
+ <element name="datefrom" type="text" minOccurs="0" maxOccurs="1"/>
+ <element name="dateuntil" type="text" minOccurs="0" maxOccurs="1"/>
+ <!-- number of words -->
+ <element name="NoW" type="text" minOccurs="0" maxOccurs="1"/>
+ <element name="corpuslink" type="link" minOccurs="0" maxOccurs="unbounded">
+ <target type="morphilo"/>
+ </element>
+ </metadata>
As a final remark, one might have noticed that all attributes are modelled as
strings although other data types are available and fields encoding the dates or
the number of words suggest otherwise. The MyCoRe framework even provides a
-data type \emph{historydate}. There is not a very satisfying answer to its
+data type *historydate*. There is not a very satisfying answer to its
disuse.
All that can be said is that the use of data types different than the string
leads later on to problems in the convergence between the search engine and the
repository framework. These issues seem to be well known and can be followed on
-github.
\ No newline at end of file
+`github <https://github.com/MyCoRe-Org>`_.
\ No newline at end of file
diff --git a/Morphilo_doc/source/framework.rst b/Morphilo_doc/source/framework.rst
deleted file mode 100644
index 1b9925de0b23715d8a34406bc9994664224637f3..0000000000000000000000000000000000000000
--- a/Morphilo_doc/source/framework.rst
+++ /dev/null
@@ -1,27 +0,0 @@
-Framework
-=========
-
-\begin{figure}
- \centering
- \includegraphics[scale=0.33]{mycore_architecture-2.png}
- \caption[MyCoRe-Architecture and Components]{MyCoRe-Architecture and Components\protect\footnotemark}
- \label{fig:abbMyCoReStruktur}
-\end{figure}
-\footnotetext{source: https://www.mycore.de}
-To specify the MyCoRe framework the morphilo application logic will have to be implemented,
-the TEI data model specified, and the input, search and output mask programmed.
-
-There are three directories which are
-important for adjusting the MyCoRe framework to the needs of one's own application. These three directories
-correspond essentially to the three components in the MVC model as explicated in
-section \ref{subsec:mvc}. Roughly, they are envisualized in figure \ref{fig:abbMyCoReStruktur} in the upper
-right hand corner. More precisely, the view (\emph{Layout} in figure \ref{fig:abbMyCoReStruktur}) and the model layer
-(\emph{Datenmodell} in figure \ref{fig:abbMyCoReStruktur}) can be done
-completely via the ``interface'', which is a directory with a predefined
-structure and some standard files. For the configuration of the logic an extra directory is offered (/src/main/java/custom/mycore/addons/). Here all, java classes
-extending the controller layer should be added.
-Practically, all three MVC layers are placed in the
-\emph{src/main/}-directory of the application. In one of the subdirectories,
-\emph{datamodel/def}, the datamodel specifications are defined as xml files. It parallels the model
-layer in the MVC pattern. How the data model was defined will be explained in
-section \ref{subsec:datamodelimpl}.
\ No newline at end of file
diff --git a/Morphilo_doc/source/images/architecture.png b/Morphilo_doc/source/images/architecture.png
new file mode 100644
index 0000000000000000000000000000000000000000..0419932563c7b07aa600982cf0117881b200c50d
Binary files /dev/null and b/Morphilo_doc/source/images/architecture.png differ
diff --git a/Morphilo_doc/source/images/morphilo_uml.png b/Morphilo_doc/source/images/morphilo_uml.png
new file mode 100644
index 0000000000000000000000000000000000000000..cac16330b005f6d7e4e8d3e35d1e00c5839fe528
Binary files /dev/null and b/Morphilo_doc/source/images/morphilo_uml.png differ
diff --git a/Morphilo_doc/source/images/mycore_architecture-2.png b/Morphilo_doc/source/images/mycore_architecture-2.png
new file mode 100644
index 0000000000000000000000000000000000000000..5e8af028e9c209ceaedb5fd8bde7d97d3ee2fc1c
Binary files /dev/null and b/Morphilo_doc/source/images/mycore_architecture-2.png differ
diff --git a/Morphilo_doc/source/view.rst b/Morphilo_doc/source/view.rst
index 5f09e06bd9d7a0d9c1edd889d8ac44d3cb36757f..5be49b3bb1ed98e41cd77c1b94439cce49e50051 100644
--- a/Morphilo_doc/source/view.rst
+++ b/Morphilo_doc/source/view.rst
@@ -4,244 +4,254 @@ View
Conceptualization
-----------------
-Lastly, the third directory (\emph{src/main/resources}) contains all code needed
+The MyCoRe-directory (*src/main/resources*) contains all code needed
for rendering the data to be displayed on the screen. So this corresponds to
the view in an MVC approach. It is done by xsl-files that (unfortunately)
contain some logic that really belongs to the controller. Thus, the division is
-not as clear as implied in theory. I will discuss this issue more specifically in the
-relevant subsection below. Among the resources are also all images, styles, and
+not as clear as implied in theory. I will point at this issue more specifically in the
+relevant subsection below. Among the resources are all images, styles, and
javascripts.
Implementation
--------------
-As explained in section \ref{subsec:mvc}, the view component handles the visual
+The view component handles the visual
representation in the form of an interface that allows interaction between
the user and the task to be carried out by the machine. As a
webservice in the present case, all interaction happens via a browser, i.e. webpages are
visualized and responses are recognized by registering mouse or keyboard
events. More specifically, a webpage is rendered by transforming xml documents
to html pages. The MyCoRe repository framework uses an open source XSLT
-processor from Apache, Xalan.\footnote{http://xalan.apache.org} This engine
+processor from Apache, `Xalan <http://xalan.apache.org>`_. This engine
transforms document nodes described by the XPath syntax into hypertext making
use of a special form of template matching. All templates are collected in so
called xml-encoded stylesheets. Since there are two data models with two
different structures, it is good practice to define two stylesheet files one for
each data model.
-As a demonstration, in listing \ref{lst:morphilostylesheet} below a short
+As a demonstration, in the listing below a short
extract is given for rendering the word data.
-\begin{lstlisting}[language=XML,caption={stylesheet
-morphilo.xsl},label=lst:morphilostylesheet]
-<?xml version="1.0" encoding="UTF-8"?>
-<xsl:stylesheet
- xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
- xmlns:xalan="http://xml.apache.org/xalan"
- xmlns:i18n="xalan://org.mycore.services.i18n.MCRTranslation"
- xmlns:acl="xalan://org.mycore.access.MCRAccessManager"
- xmlns:mcr="http://www.mycore.org/" xmlns:xlink="http://www.w3.org/1999/xlink"
- xmlns:mods="http://www.loc.gov/mods/v3"
- xmlns:encoder="xalan://java.net.URLEncoder"
- xmlns:mcrxsl="xalan://org.mycore.common.xml.MCRXMLFunctions"
- xmlns:mcrurn="xalan://org.mycore.urn.MCRXMLFunctions"
- exclude-result-prefixes="xalan xlink mcr i18n acl mods mcrxsl mcrurn encoder"
- version="1.0">
- <xsl:param name="MCR.Users.Superuser.UserName"/>
-
- <xsl:template match="/mycoreobject[contains(@ID,'_morphilo_')]">
- <head>
- <link href="{$WebApplicationBaseURL}css/file.css" rel="stylesheet"/>
- </head>
- <div class="row">
- <xsl:call-template name="objectAction">
- <xsl:with-param name="id" select="@ID"/>
- <xsl:with-param name="deriv" select="structure/derobjects/derobject/@xlink:href"/>
- </xsl:call-template>
- <xsl:variable name="objID" select="@ID"/>
- <!-- Hier Ueberschrift setzen -->
- <h1 style="text-indent: 4em;">
- <xsl:if test="metadata/def.morphiloContainer/morphiloContainer/morphilo/w">
- <xsl:value-of select="metadata/def.morphiloContainer/morphiloContainer/morphilo/w/text()[string-length(normalize-space(.))>0]"/>
- </xsl:if>
- </h1>
- <dl class="dl-horizontal">
- <!-- (1) Display word -->
- <xsl:if test="metadata/def.morphiloContainer/morphiloContainer/morphilo/w">
- <dt>
- <xsl:value-of select="i18n:translate('response.page.label.word')"/>
- </dt>
- <dd>
- <xsl:value-of select="metadata/def.morphiloContainer/morphiloContainer/morphilo/w/text()[string-length(normalize-space(.))>0]"/>
- </dd>
- </xsl:if>
- <!-- (2) Display lemma -->
- ...
- </xsl:template>
- ...
- <xsl:template name="objectAction">
- ...
- </xsl:template>
-...
-</xsl:stylesheet>
-\end{lstlisting}
+.. code-block:: xml
+ :caption: word data rendering in morphilo.xsl
+ :name: morphilo.xsl
+
+ <?xml version="1.0" encoding="UTF-8"?>
+ <xsl:stylesheet
+ xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
+ xmlns:xalan="http://xml.apache.org/xalan"
+ xmlns:i18n="xalan://org.mycore.services.i18n.MCRTranslation"
+ xmlns:acl="xalan://org.mycore.access.MCRAccessManager"
+ xmlns:mcr="http://www.mycore.org/" xmlns:xlink="http://www.w3.org/1999/xlink"
+ xmlns:mods="http://www.loc.gov/mods/v3"
+ xmlns:encoder="xalan://java.net.URLEncoder"
+ xmlns:mcrxsl="xalan://org.mycore.common.xml.MCRXMLFunctions"
+ xmlns:mcrurn="xalan://org.mycore.urn.MCRXMLFunctions" exclude-result-prefixes="xalan xlink mcr i18n acl mods mcrxsl mcrurn encoder" version="1.0">
+ <xsl:param name="MCR.Users.Superuser.UserName"/>
+ <xsl:template match="/mycoreobject[contains(@ID,'_morphilo_')]">
+ <head>
+ <link href="{$WebApplicationBaseURL}css/file.css" rel="stylesheet"/>
+ </head>
+ <div class="row">
+ <xsl:call-template name="objectAction">
+ <xsl:with-param name="id" select="@ID"/>
+ <xsl:with-param name="deriv" select="structure/derobjects/derobject/@xlink:href"/>
+ </xsl:call-template>
+ <xsl:variable name="objID" select="@ID"/>
+ <!-- Hier Ueberschrift setzen -->
+ <h1 style="text-indent: 4em;">
+ <xsl:if test="metadata/def.morphiloContainer/morphiloContainer/morphilo/w">
+ <xsl:value-of select="metadata/def.morphiloContainer/morphiloContainer/morphilo/w/text()[string-length(normalize-space(.))>0]"/>
+ </xsl:if>
+ </h1>
+ <dl class="dl-horizontal">
+ <!-- (1) Display word -->
+ <xsl:if test="metadata/def.morphiloContainer/morphiloContainer/morphilo/w">
+ <dt>
+ <xsl:value-of select="i18n:translate('response.page.label.word')"/>
+ </dt>
+ <dd>
+ <xsl:value-of select="metadata/def.morphiloContainer/morphiloContainer/morphilo/w/text()[string-length(normalize-space(.))>0]"/>
+ </dd>
+ </xsl:if>
+ <!-- (2) Display lemma -->
+ ...
+ </xsl:template>
+ ...
+ <xsl:template name="objectAction">
+ ...
+ </xsl:template>
+ ...
+ </xsl:stylesheet>
+
+
This template matches with
-the root node of each \emph{MyCoRe object} ensuring that a valid MyCoRe model is
+the root node of each *MyCoRe object* ensuring that a valid MyCoRe model is
used and checking that the document to be processed contains a unique
-identifier, here a \emph{MyCoRe-ID}, and the name of the correct data model,
-here \emph{morphilo}.
-Then, another template, \emph{objectAction}, is called together with two parameters, the ids
+identifier, here a *MyCoRe-ID*, and the name of the correct data model,
+here *morphilo*.
+Then, another template, *objectAction*, is called together with two parameters, the ids
of the document object and attached files. In the remainder all relevant
information from the document is accessed by XPath, such as the word and the lemma,
and enriched with hypertext annotations it is rendered as a hypertext document.
-The template \emph{objectAction} is key to understand the coupling process in the software
-framework. It is therefore separately listed in \ref{lst:objActionTempl}.
-
-\begin{lstlisting}[language=XML,caption={template
-objectAction},label=lst:objActionTempl,escapechar=|]
-<xsl:template name="objectAction">
- <xsl:param name="id" select="./@ID"/>
- <xsl:param name="accessedit" select="acl:checkPermission($id,'writedb')"/>
- <xsl:param name="accessdelete" select="acl:checkPermission($id,'deletedb')"/>
- <xsl:variable name="derivCorp" select="./@label"/>
- <xsl:variable name="corpID" select="metadata/def.corpuslink[@class='MCRMetaLinkID']/corpuslink/@xlink:href"/>
- <xsl:if test="$accessedit or $accessdelete">|\label{ln:ng}|
- <div class="dropdown pull-right">
- <xsl:if test="string-length($corpID) > 0 or $CurrentUser='administrator'">
- <button class="btn btn-default dropdown-toggle" style="margin:10px" type="button" id="dropdownMenu1" data-toggle="dropdown" aria-expanded="true">
- <span class="glyphicon glyphicon-cog" aria-hidden="true"></span> Annotieren
- <span class="caret"></span>
- </button>
- </xsl:if>
- <xsl:if test="string-length($corpID) > 0">|\label{ln:ru}|
- <xsl:variable name="ifsDirectory" select="document(concat('ifs:/',$derivCorp))"/>
- <ul class="dropdown-menu" role="menu" aria-labelledby="dropdownMenu1">
- <li role="presentation">
- |\label{ln:nw1}|<a href="{$ServletsBaseURL}object/tag{$HttpSession}?id={$derivCorp}&objID={$corpID}" role="menuitem" tabindex="-1">|\label{ln:nw2}|
- <xsl:value-of select="i18n:translate('object.nextObject')"/>
- </a>
- </li>
- <li role="presentation">
- <a href="{$WebApplicationBaseURL}receive/{$corpID}" role="menuitem" tabindex="-1">
- <xsl:value-of select="i18n:translate('object.backToProject')"/>
- </a>
- </li>
- </ul>
- </xsl:if>
- <xsl:if test="$CurrentUser='administrator'">
- <ul class="dropdown-menu" role="menu" aria-labelledby="dropdownMenu1">
- <li role="presentation">
- <a role="menuitem" tabindex="-1" href="{$WebApplicationBaseURL}content/publish/morphilo.xed?id={$id}">
- <xsl:value-of select="i18n:translate('object.editWord')"/>
- </a>
- </li>
- <li role="presentation">
- <a href="{$ServletsBaseURL}object/delete{$HttpSession}?id={$id}" role="menuitem" tabindex="-1" class="confirm_deletion option" data-text="Wirklich loeschen">
- <xsl:value-of select="i18n:translate('object.delWord')"/>
- </a>
- </li>
- </ul>
- </xsl:if>
- </div>
- <div class="row" style="margin-left:0px; margin-right:10px">
- <xsl:apply-templates select="structure/derobjects/derobject[acl:checkPermission(@xlink:href,'read')]">
- <xsl:with-param name="objID" select="@ID"/>
- </xsl:apply-templates>
- </div>
- </xsl:if>
-</xsl:template>
-\end{lstlisting}
-The \emph{objectAction} template defines the selection menu appearing -- once manual tagging has
+The template *objectAction* is key to understand the coupling process in the software
+framework. It is therefore separately listed in :ref:`objActionTempl`.
+
+.. _objActionTempl:
+
+.. code-block:: xml
+ :caption: template ObjectAction
+ :linenos:
+ :emphasize-lines: 7, 15, 19
+
+ <xsl:template name="objectAction">
+ <xsl:param name="id" select="./@ID"/>
+ <xsl:param name="accessedit" select="acl:checkPermission($id,'writedb')"/>
+ <xsl:param name="accessdelete" select="acl:checkPermission($id,'deletedb')"/>
+ <xsl:variable name="derivCorp" select="./@label"/>
+ <xsl:variable name="corpID" select="metadata/def.corpuslink[@class='MCRMetaLinkID']/corpuslink/@xlink:href"/>
+ <xsl:if test="$accessedit or $accessdelete">
+ <div class="dropdown pull-right">
+ <xsl:if test="string-length($corpID) > 0 or $CurrentUser='administrator'">
+ <button class="btn btn-default dropdown-toggle" style="margin:10px" type="button" id="dropdownMenu1" data-toggle="dropdown" aria-expanded="true">
+ <span class="glyphicon glyphicon-cog" aria-hidden="true"></span> Annotieren
+ <span class="caret"></span>
+ </button>
+ </xsl:if>
+ <xsl:if test="string-length($corpID) > 0">
+ <xsl:variable name="ifsDirectory" select="document(concat('ifs:/',$derivCorp))"/>
+ <ul class="dropdown-menu" role="menu" aria-labelledby="dropdownMenu1">
+ <li role="presentation">
+ <a href="{$ServletsBaseURL}object/tag{$HttpSession}?id={$derivCorp}&objID={$corpID}" role="menuitem" tabindex="-1">
+ <xsl:value-of select="i18n:translate('object.nextObject')"/>
+ </a>
+ </li>
+ <li role="presentation">
+ <a href="{$WebApplicationBaseURL}receive/{$corpID}" role="menuitem" tabindex="-1">
+ <xsl:value-of select="i18n:translate('object.backToProject')"/>
+ </a>
+ </li>
+ </ul>
+ </xsl:if>
+ <xsl:if test="$CurrentUser='administrator'">
+ <ul class="dropdown-menu" role="menu" aria-labelledby="dropdownMenu1">
+ <li role="presentation">
+ <a role="menuitem" tabindex="-1" href="{$WebApplicationBaseURL}content/publish/morphilo.xed?id={$id}">
+ <xsl:value-of select="i18n:translate('object.editWord')"/>
+ </a>
+ </li>
+ <li role="presentation">
+ <a href="{$ServletsBaseURL}object/delete{$HttpSession}?id={$id}" role="menuitem" tabindex="-1" class="confirm_deletion option" data-text="Wirklich loeschen">
+ <xsl:value-of select="i18n:translate('object.delWord')"/>
+ </a>
+ </li>
+ </ul>
+ </xsl:if>
+ </div>
+ <div class="row" style="margin-left:0px; margin-right:10px">
+ <xsl:apply-templates select="structure/derobjects/derobject[acl:checkPermission(@xlink:href,'read')]">
+ <xsl:with-param name="objID" select="@ID"/>
+ </xsl:apply-templates>
+ </div>
+ </xsl:if>
+ </xsl:template>
+
+The *objectAction* template defines the selection menu appearing -- once manual tagging has
started -- on the upper right hand side of the webpage entitled
-\emph{Annotieren} and displaying the two options \emph{next word} or \emph{back
-to project}.
-The first thing to note here is that in line \ref{ln:ng} a simple test
+*Annotieren* and displaying the two options *next word* or *back
+to project*.
+The first thing to note here is that in line 7 a simple test
excludes all guest users from accessing the procedure. After ensuring that only
-the user who owns the corpus project has access (line \ref{ln:ru}), s/he will be
+the user who owns the corpus project has access (line 15), s/he will be
able to access the drop down menu, which is really a url, e.g. line
-\ref{ln:nw1}. The attentive reader might have noticed that
-the url exactly matches the definition in the web-fragment.xml as shown in
-listing \ref{lst:webfragment}, line \ref{ln:tag}, which resolves to the
+19. The attentive reader might have noticed that
+the url exactly matches the definition in the *web-fragment.xml* as shown in
+listing :ref:`webxml`, line 17, which resolves to the
respective java class there. Really, this mechanism is the data interface within the
-MVC pattern. The url also contains two variables, named \emph{derivCorp} and
-\emph{corpID}, that are needed to identify the corpus and file object by the
-java classes (see section \ref{sec:javacode}).
+MVC pattern. The url also contains two variables, named *derivCorp* and
+*corpID*, that are needed to identify the corpus and file object by the
+java classes (see section :ref:`controller-section`).
The morphilo.xsl stylesheet contains yet another modification that deserves mention.
-In listing \ref{lst:derobjectTempl}, line \ref{ln:morphMenu}, two menu options --
-\emph{Tag automatically} and \emph{Tag manually} -- are defined. The former option
-initiates ProcessCorpusServlet.java as can be seen again in listing \ref{lst:webfragment},
-line \ref{ln:process}, which determines words that are not in the master data base.
+In listing :ref:`derobjectTempl`, line 18, two menu options --
+*Tag automatically* and *Tag manually* -- are defined. The former option
+initiates ProcessCorpusServlet.java as can be seen again in listing :ref:`webxml`,
+line 7, which determines words that are not in the master data base.
Still, it is important to note that the menu option is only displayed if two restrictions
-are met. First, a file has to be uploaded (line \ref{ln:1test}) and, second, there must be
+are met. First, a file has to be uploaded (line 19) and, second, there must be
only one file. This is necessary because in the annotation process other files will be generated
that store the words that were not yet processed or a file that includes the final result. The
generated files follow a certain pattern. The file harboring the final, entire TEI-annotated
-corpus is prefixed by \emph{tagged}, the other file is prefixed \emph{untagged}. This circumstance
-is exploited for manipulating the second option (line \ref{ln:loop}). A loop runs through all
-files in the respective directory and if a file name starts with \emph{untagged},
+corpus is prefixed by *tagged*, the other file is prefixed *untagged*. This circumstance
+is exploited for manipulating the second option (line 27). A loop runs through all
+files in the respective directory and if a file name starts with *untagged*,
the option to manually tag is displayed.
-\begin{lstlisting}[language=XML,caption={template
-matching derobject},label=lst:derobjectTempl,escapechar=|]
-<xsl:template match="derobject" mode="derivateActions">
- <xsl:param name="deriv" />
- <xsl:param name="parentObjID" />
- <xsl:param name="suffix" select="''" />
- <xsl:param name="id" select="../../../@ID" />
- <xsl:if test="acl:checkPermission($deriv,'writedb')">
- <xsl:variable name="ifsDirectory" select="document(concat('ifs:',$deriv,'/'))" />
- <xsl:variable name="path" select="$ifsDirectory/mcr_directory/path" />
- ...
- <div class="options pull-right">
- <div class="btn-group" style="margin:10px">
- <a href="#" class="btn btn-default dropdown-toggle" data-toggle="dropdown">
- <i class="fa fa-cog"></i>
- <xsl:value-of select="' Korpus'"/>
- <span class="caret"></span>
- </a>
- <ul class="dropdown-menu dropdown-menu-right">
- <!-- Anpasssungen Morphilo -->|\label{ln:morphMenu}|
- <xsl:if test="string-length($deriv) > 0">|\label{ln:1test}|
- <xsl:if test="count($ifsDirectory/mcr_directory/children/child) = 1">|\label{ln:2test}|
- <li role="presentation">
- <a href="{$ServletsBaseURL}object/process{$HttpSession}?id={$deriv}&objID={$id}" role="menuitem" tabindex="-1">
- <xsl:value-of select="i18n:translate('derivate.process')"/>
- </a>
- </li>
- </xsl:if>
- <xsl:for-each select="$ifsDirectory/mcr_directory/children/child">|\label{ln:loop}|
- <xsl:variable name="untagged" select="concat($path, 'untagged')"/>
- <xsl:variable name="filename" select="concat($path,./name)"/>
- <xsl:if test="starts-with($filename, $untagged)">
- <li role="presentation">
- <a href="{$ServletsBaseURL}object/tag{$HttpSession}?id={$deriv}&objID={$id}" role="menuitem" tabindex="-1">
- <xsl:value-of select="i18n:translate('derivate.taggen')"/>
- </a>
- </li>
+.. _derobjectTempl:
+
+.. code-block:: xml
+ :caption: derobject template
+ :linenos:
+ :emphasize-lines: 18,19, 27
+
+ <xsl:template match="derobject" mode="derivateActions">
+ <xsl:param name="deriv" />
+ <xsl:param name="parentObjID" />
+ <xsl:param name="suffix" select="''" />
+ <xsl:param name="id" select="../../../@ID" />
+ <xsl:if test="acl:checkPermission($deriv,'writedb')">
+ <xsl:variable name="ifsDirectory" select="document(concat('ifs:',$deriv,'/'))" />
+ <xsl:variable name="path" select="$ifsDirectory/mcr_directory/path" />
+ ...
+ <div class="options pull-right">
+ <div class="btn-group" style="margin:10px">
+ <a href="#" class="btn btn-default dropdown-toggle" data-toggle="dropdown">
+ <i class="fa fa-cog"></i>
+ <xsl:value-of select="' Korpus'"/>
+ <span class="caret"></span>
+ </a>
+ <ul class="dropdown-menu dropdown-menu-right">
+ <!-- Anpasssungen Morphilo -->|\label{ln:morphMenu}|
+ <xsl:if test="string-length($deriv) > 0">|\label{ln:1test}|
+ <xsl:if test="count($ifsDirectory/mcr_directory/children/child) = 1">|\label{ln:2test}|
+ <li role="presentation">
+ <a href="{$ServletsBaseURL}object/process{$HttpSession}?id={$deriv}&objID={$id}" role="menuitem" tabindex="-1">
+ <xsl:value-of select="i18n:translate('derivate.process')"/>
+ </a>
+ </li>
+ </xsl:if>
+ <xsl:for-each select="$ifsDirectory/mcr_directory/children/child">|\label{ln:loop}|
+ <xsl:variable name="untagged" select="concat($path, 'untagged')"/>
+ <xsl:variable name="filename" select="concat($path,./name)"/>
+ <xsl:if test="starts-with($filename, $untagged)">
+ <li role="presentation">
+ <a href="{$ServletsBaseURL}object/tag{$HttpSession}?id={$deriv}&objID={$id}" role="menuitem" tabindex="-1">
+ <xsl:value-of select="i18n:translate('derivate.taggen')"/>
+ </a>
+ </li>
+ </xsl:if>
+ </xsl:for-each>
</xsl:if>
- </xsl:for-each>
- </xsl:if>
- ...
- </ul>
- </div>
- </div>
- </xsl:if>
-</xsl:template>
-\end{lstlisting}
-
-Besides the two stylesheets morphilo.xsl and corpmeta.xsl, other stylesheets have
+ ...
+ </ul>
+ </div>
+ </div>
+ </xsl:if>
+ </xsl:template>
+
+
+Besides the two stylesheets *morphilo.xsl* and *corpmeta.xsl*, other stylesheets have
to be adjusted. They will not be discussed in detail here for they are self-explanatory for the most part.
-Essentially, they render the overall layout (\emph{common-layout.xsl}, \emph{skeleton\_layout\_template.xsl})
+Essentially, they render the overall layout (*common-layout.xsl*, *skeleton_layout_template.xsl*)
or the presentation
-of the search results (\emph{response-page.xsl}) and definitions of the solr search fields (\emph{searchfields-solr.xsl}).
-The former and latter also inherit templates from \emph{response-general.xsl} and \emph{response-browse.xsl}, in which the
+of the search results (*response-page.xsl*) and definitions of the solr search fields (*searchfields-solr.xsl*).
+The former and latter also inherit templates from *response-general.xsl* and *response-browse.xsl*, in which the
navigation bar of search results can be changed. For the use of multilinguality a separate configuration directory
-has to be created containing as many \emph{.property}-files as different
-languages want to be displayed. In the current case these are restricted to German and English (\emph{messages\_de.properties} and \emph{messages\_en.properties}).
-The property files include all \emph{i18n} definitions. All these files are located in the \emph{resources} directory.
+has to be created containing as many *.property*-files as different
+languages want to be displayed. In the current case these are restricted to German and English (*messages_de.properties* and *messages_en.properties*).
+The property files include all *i18n* definitions. All these files are located in the *resources* directory.
Furthermore, a search mask and a page for manually entering the annotations had
to be designed.
-For these files a specially designed xml standard (\emph{xed}) is recommended to be used within the
+For these files a specially designed xml standard (*xed*) is recommended to be used within the
repository framework.
\ No newline at end of file