\documentclass{article}
%\usepackage{graphicx,multicol,tabularx,epsfig}
%%\usepackage{lingmacros}
%\usepackage{wrapfig}


\usepackage{epsfig}
\usepackage{graphicx,wrapfig}
%\usepackage{lingmacros} % avm, tree-dvips}
%%\usepackage[dvips]{color}
\usepackage{latexsym}
\usepackage{amssymb}
%\usepackage{listings}
\usepackage{pstricks}
\usepackage{boxedminipage}

%%%%%%%Eric's
%2007% \usepackage[T1]{fontenc}
%\usepackage{times}
%%\usepackage{listings}
\usepackage{url}
%%\usepackage{avm}

%\usepackage[leqno,centerstructures]{amsmath}
\usepackage[leqno]{amsmath}
\usepackage{amsfonts}

\def\Feat{\mathrm{\bf Feat}}
\def\Atom{\mathrm{\bf Atom}}
\def\Type{\mathrm{\bf Type}}
\def\Var{\mathrm{\bf Var}}

\def\feat{\mathrm{feat}}
\def\atom{\mathrm{atom}}
\def\type{\mathrm{type}}
\def\var{\mathrm{var}}
\def\dom{\mathrm{dom}}

\def\Category{\mathcal{U}}
\def\Int{\mathcal{I}}

\newcommand{\EQUIV}{\Longleftrightarrow}
\newcommand{\IMP}{\Rightarrow}
\newcommand{\OR}{{\rm or}}
\newcommand{\AND}{{\rm and}}

\def\FS{\mathcal{FS}}

\def\coll#1#2{{}_{\mathrm{#1}}\{#2\}}
\def\List#1{\coll{list}{#1}}
\def\Bag#1{\coll{bag}{#1}}
\def\Set#1{\coll{set}{#1}}
\def\Fs#1#2{{}_{#1}[#2]}
\def\Not#1{\neg#1}

\def\oubli#1{}



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\def\ignore#1{}
\def\meta#1{\comments{META: #1}}
\def\question#1{{\small\textbf{QUESTION:} #1}}
\def\elt#1{\texttt{#1}}
\def\attr#1{\texttt{#1}}

\let\hypertarget\ignore

%\iffalse
\newcommand{\comments}[1]%
        {\bigskip\noindent\begin{boxedminipage}{152mm}%
        \textcolor[rgb]{1.00,0.00,0.00}{{\tt #1 }}%
        \end{boxedminipage}\bigskip%
        }
\newcommand{\editor}[1]%
        {\bigskip\noindent\begin{boxedminipage}{152mm}%
        \textcolor{blue}{{\tt #1 }}%
        \end{boxedminipage}\bigskip%
        }

\newcommand{\strShare}[1]{\fbox{\scriptsize \(#1\)}}
\newcommand{\cll}[1]{\begin{minipage}[t]{50mm}\hspace{0pt} #1 \vspace{2mm}\end{minipage}}
\newcommand{\clr}[1]{\begin{minipage}[t]{100mm}\hspace{0pt} #1 \vspace{2mm}\end{minipage}}
\newcommand{\hr}[1]{\begin{minipage}[t]{10mm}\hspace{0pt}#1\vspace{0,2cm}\end{minipage}}
%\fi


\begin{document}




\title{Defining semantic annotation languages; the case of ISO-TimeML}% working title
\author{ Harry Bunt and Chwhynny Overbeeke\\
%\small Department of Language and Information Science,\\
%\small Faculty of Arts, Tilburg University\\
%\small P.O. Box 90153, 5000 LE Tilburg, Netherlands\\
{\footnotesize  harry.bunt@uvt.nl, info@chwhynny.com}}

\date{}

\maketitle

%\begin{abstract}
%\noindent This paper has no abstract.
%\end{abstract}

\section{Introduction}
In recent years, expert groups within the international
organization for standards ISO have started up various projects,
concerned with the definition of interoperable concepts for
annotation, with the ultimate aim to support the development of
interoperable language resources.
% growing interest in the standardization of semantic annotation to enable the exchange and reuse of multilingual language resources. The research described in this paper has been carried out within the LIRICS\footnote{{\b L}inguistic {\b I}nf{\b R}astructure for {\b I}nteroperable Resour{\b C}es and {\b S}ystems (http://lirics.loria.fr)} project, which aims towards providing standards for different areas of language technology in collaboration with ISO TC37/SC4, one of which is semantic role annotation. We aim (1) to identify commonalities and differences in alternative approaches to the annotation and representation of semantic role information; and (2) to develop methodological principles for characterising well-defined concepts for semantic roles.

The Linguistic Annotation Framework (LAF, Ide \& et al., 2003) %, a proposed ISO standard,
thereby serves as a meta-framework for efforts in syntactic annotation,  
% (the `Syntactic Annotation Framework' SynAF; the `Morphosyntactic Annotation Framework' (MAF); and the `Semantic Annotation Framework' (SemAF). 
in morphosyntactic annotation, and in semantic annotation. 
LAF makes a fundamental distinction between the concepts of  {\it annotation} and {\it representation}. 
The term `annotation' is used to refer to the process of adding information to segments of language data, 
or to refer to that information itself. 
This notion is independent of the format in which this information is represented. 
The term `representation' is used to refer to the format in which an annotation is rendered, for instance in XML. %, independent of its content. 
%\& et al. note that the \&ntification of a segment in the language data is in fact also a part of an annotation task.
According to LAF, {\it annotations} are the proper level of standardization, not representations.  
The proposed ISo standard for temporal annotation 
%`SemAF Part 1, Time and Events' 
(ISO SemAF/Time, 2008) therefore defines a markup language 
% for annotating documents with information about time and events 
at the level of annotations. This language is called ISO-TimeML.\ \\


The distinction between annotations and representations is
reflected in the specification of ISO-TimeML given below, where an
{\it abstract syntax} is defined as well as a {\it concrete
syntax}. The abstract  syntax specifies the elements making up the
information in annotations, and how these elements may be combined
to form complex annotation structures; these combinations are
defined as set-theoretical structures, independent of any
particular representation format. The ISO-TimeML proposal includes
an XML-based concrete syntax, which is a variant of the TimeML
syntax (Pustejovsky et al., 2003). %; 2007). 
Any other representation
that is a faithful rendering of the abstract syntax of ISO-TimeML
can readily be converted into this XML representation and vice
versa. ISO-TimeML has a semantics associated with its abstract
syntax, which defines the meanings of ISO-TimeML annotation
structures. The fact that this semantics is associated with the
{\it abstract} syntax, rather than with a particular concrete
syntax, explains why all concrete representations of ISO-TimeML
annotations are semantically equivalent.\\


In this note we argue that the distinction of an abstract and a concrete syntax level is not only desirable from a standardization point of view, but can also be beneficial for designing semantic annotation languages with a representation format that is conceptually transparent for annotators and that allows a simple, systematic interpretation. We illustrate this for the annotation and interpretation of expressions denoting dates, times, and durations.


\subsection{ISO-TimeML}
The abstract syntax of ISO-TimeML defines the set-theoretical structures %(``ISO-TimeML-structures'')
that constitute the information about time and events that may be
contained in annotations. The  abstract syntax
consists of two parts: (a) a `conceptual
inventory', specifying the elements from
which these structures are built up; and (b) a set of  syntax rules which describe the
possible combinations of these elements. 
%What these combinations mean, i.e. which information they capture, is specified by the semantics associated with the abstract syntax.

\ \\
{\bf a. Conceptual inventory}
The concepts that can be used to build ISO-TimeML annotations fall into the following five categories, all formed by finite sets, plus the concepts of real and natural numbers. %Natural numbers are needed for capturing the information expressed in English by {\it ``twice"} and {\it ``three times"}; real numbers are needed for cases such as {\it `two and a half hours"}.

 \begin{itemize}
   \item finite sets of elements called `event classes';
   `tenses', `aspects', `polarities', and `set-theoretic types' ; %`verb forms'; `syntactic moods', `syntactic modalities',
   \item finite sets of elements called
   `temporal %anchoring
   relations',%`temporal ordering relations',  `temporal measurement functions',
   `duration relations', `numerical relations, `event subordination relations', and `aspectual relations';
   \item a finite set of elements called `time zones';
   \item finite sets of elements called `calendar years', `calendar months', `calendar day numbers';
   `clock times'; % (natural numbers ranging from 0000 to 0059; from 0100 to 0159; ... from 2300 to 2400);
   \item a finite set of elements called `temporal units'.
%   \item the sets of natural  numbers and real numbers.

  \end{itemize}

\ \\
% \item
{\bf b. Syntax rules}
Annotation structures in ISO-TimeML come in two varieties, {\it entity structures} and {\it link structures}. 
Entity structures contain semantic information about a segment of source text; link structures describe semantic relations 
between segments of source text. 
%which are not made explicit in the text; they do so by means of links between entity structures. \\

The simplest kind of ISO-TimeML structures are a single entity structure, which is a pair $<m, a>$
 consisting of a markable\footnote{The
   generic term {\it markable} is used to refer to the entities that the annotations are associated with.
   There are two kinds of markables in ISO-TimeML: {\it event markables} and {\it time markables},
    corresponding to segments of primary data   that describe events, and to those that describe temporal
     entities or relations, respectively.}
$m$ and an annotation $a$, or a single link structure $<e_1, e_2, R>$ which relates two entity structures.  More complex annotation structures consist of a set of  entity structures and a set of link structures which link the entity structures together. 
% through temporal and inter-event relations.


%More formally, an ISO-TimeML annotation structure consists of two sets $M$ and $L$, where $M$ is a set of pairs
%$<$$markable$, $entity$ $structure$$>$ and $L$ is a set of triples
%$<$$entity$ $structure$, $entity$ $structure$, $link$ $structure$$>$, such that each element of $L$ contains at least one entity structure that occurs in $M$.
%(This latter condition ensures that the links in an annotation structure relate to the entity structures that it contains; they may in addition also contain links to elements outside the current annotation structure).

\ \\
{\it Entity structures} come in 6 types,
containing information about (1) events; (2) temporal intervals;
(3) time points (``instants''); (4) amounts of time; (5) frequencies of events;
and (6) temporal relations.
% (as for instance expressed by temporal prepositions). 
We focus here on the tree types of temporal
concepts: intervals, instants, and amounts of time.

\begin{enumerate}

%   \item An {\it event structure} is a 7-tuple $<C, T, A,  \Sigma, N, P_N, V>$

   \item An  {\it instant structure} is either
     a triple $<time$ $zone$, $date$, $clocktime>$, where a $date$ is a triple consisting of a calendar
     year, a calendar month, and a calendar day number;   or a triple
     $<time$-$amount$ $structure$, $instant$ $structure$, $temporal$ $relation$$>$
     ({\it ``half an hour before midnight"}).
   \item An  {\it interval structures}is either:
     \begin{enumerate}
      \item a pair $<t_1, t_2>$ of two instant structures %, corresponding to the 
       (beginning and end); 
       % points of the interval;
      \item a calendar year ({\it ``1995''}), 
        a pair consisting of a calendar year and a calendar month ({\it ``March 1995''}), 
        or a triple $<cal.year, cal. month, cal.daynumber>$ ({\it ``March 16 1995''});
      \item a triple $<time$-$amount$ $structure$, $interval$ $structure$, $temporal$ $relation>$
       ({``three weeks before Christmas"}); % ``two years from today"});
      \item a triple $<t_1, t_2, R>$ where $t_1$ and $t_2$ are either instant structures or interval
       structures, and where $R$ is a duration relation ({\it ``from 1992 until 1995"}).
       %; ``from 1882 through 1995"})
     \end{enumerate}

   \item A {\it time-amount structure} is a pair $<n, u>$, where $n$
   is a real number and $u$ a temporal unit, or a triple $<R, n, u>$, where $R$ is a numerical relation
   (like {\it greater than}) and $n$ and $u$ as before;

%   \item A {\it multiplicity structure} is a triple
%   $<numerical$ $relation, number, time$-$amount$ $structure>$

%   \item A {\it temporal relation structure} is just a temporal relation.

\end{enumerate}

\noindent
{\it Link structures} serve to specify the temporal anchoring of events in time; the temporal ordering of events, intervals or instants; the  length of an interval;  
subordination relations between events; and aspectual relations between events.
\\


% \subsection{Semantics}
The semantics associated with this abstract syntax defines a mapping from the set-theoretical structures 
defined by the abstact syntax to the language of first-order predicate logic with lambda abstraction. 
This semantics has been inspired by the compositional translation, defined in Bunt \& Overbeeke (2008), 
from a concrete XML syntax of ISO-TimeML  
%in the style of the underlying TimeML language (Pustejovsky et al., 2003) %; 2007) 
to first-order logic.
%Given that first-order logic has a well-established compositional formal semantics, this approach defines a compositional semantics for ISO-TimeML.



%The semantics of this annotation structure can be computed by combining the first-order logic representations of the subordination structure and the temporal anchoring structure (see Bunt \& Overbeeke, 2008 for how to do this), with the result: \\
%\ \\
%$\exists e_1:$ {\sc process}($e_1$ $\wedge$ $\exists t_1:$ calyear({\sc cet}, $t_1$)=2008 $\wedge$
%clocktime({\sc cet}($t_1$))=1045 $\wedge$ {\sc ev-time}({\sc cet}($e_1$))=$t_1$ $\wedge$
%$\exists e_2:$ {\sc initiates}($e_1, e_2$)\\



% \subsection{Concrete syntax}
A concrete syntax consists of the specification of names for the various sets that form the conceptual vocabulary, plus a listing of specific named elements of these sets, and for each rule of the abstract syntax a specification of how to represent the constructed annotation structure. \\
\ \\
A particular XML-based syntax for temporal annotation has been defined in the TimeML effort (Pustejovsky et al., 2003). 


This syntax makes use of a TIMEX3 tag to mark up explicit temporal
expressions like dates, times and durations. In this tag, the
different types of temporal expressions are represented by means
of the attribute {\small \texttt{type}}. There is also an
attribute called {\small \texttt{value}} which is presented in a
standard format that consist of a combination of calendar days,
weeks, months and years or a combination thereof ({\small
\texttt{2007-03-16}}), clock hours, minutes and seconds or a
combination thereof ({\small \texttt{T13:15:00}}), or durations of
a certain length and type ({\small \texttt{P60D}}). Sometimes
there is a modifier. This way of representation does not account
for the conceptual differences between the different types of
temporal expressions. In order to do this, we introduce new tags
that match their conceptual differences. We also introduce a time
zone attribute, as this was not accounted for in the ISO-TimeML
TIMEX3 tag. As a result we have a simple interpretation that
translates annotations to First Order Logic. In most cases,
temporal information is incomplete. In the following examples of
our new tags we put the unused optional attributes between square
brackets.\\

\noindent 1. The expression {\it ``March 2007"} would be represented as
follows:
\begin{verbatim}
<TIMEX3 id=''t1'' type=''DATE'' value=''2006-03-XX> 
<DATE id="t1" calyear="2007" calmonth="03" [calweek=WW] calday="16"
      [mod=MOD]>
\end{verbatim}
$\lambda t. $interval(t) $\wedge$ calyear(t)=2007 $\wedge$ calmonth=march \\
\ \\
\noindent The time {\it ``1:35 p.m. CET"} would be represented as
follows:
\begin{verbatim}
<TIMEX3 id=''t1'' type=''TIME'' alue=''T13:35''>
<TIME id="t" clockTime="1335" timezone="CET">
\end{verbatim}
$\lambda t. $interval(t) $\wedge$ calyear(t)=2007 $\wedge$ calmonth=march \\
\ \\

\noindent The duration {\it ``no more than thirty minutes"} would be represented as follows:
\begin{verbatim}
<TIMEX3 id=''t1'' type=''DURATION'' value=''P30M''> 
<TIMEAMOUNT id="u" type="d1" value="30" mod="EQUAL_OR_LESS">
\end{verbatim}
$\lambda t. $interval(t) $\wedge$ calyear(t)=2007 $\wedge$ calmonth=march \\




\section{Conclusions}


\begin{thebibliography}{99}
\small
\bibitem{BuntOverbeeke}Bunt, H.C,  Overbeeke, C. (2008) \emph{An Extensible Compositional Semantics for Temporal Annotation.} In: N. \& et al., {\it Proceedings of LAW II, the Second Workshop on Linguistic Annotation}, Marrakech, May 2008.

\bibitem{Bunt} Bunt, H.C., Romary, L. (2002) \emph{Requirements on multimodal semantic representations.} Proceedings of ISO TC37/SC4 Preliminary Meeting, Choi, K.S. (eds.). Dordrecht, Seoul, 59-68. \vspace{-2mm}
\bibitem{Setzer} Setzer, A. (2001) \emph{Temporal Information in Newswire Articles; An Annotation Scheme and Corpus Study.} PhD Thesis, University of Sheffield.
\bibitem{Ferro} Ferro, L., Gerber, L., Mani, I., Sundheim, B. and Wilson, G. (2002) \emph{Instruction Manual for the Annotation of Temporal Expressions.} MITRE Washington C3 Center, McLean, Virginia.
%\bibitem{Dowty91} Dowty, D. (1991). \emph{Thematic Proto-Roles and Argument Selection}. Language 67, 547-619. \vspace{-2mm}
%\bibitem{FrameNet} ICSI, Berkley (2005). \emph{FrameNet}. http://framenet.icsi.berkeley.edu. \vspace{-2mm}
%\bibitem{sil} Loos, E.E., Anderson, S., Dwight, H. (2004). \emph{Glossary of linguistic terms}. http://www.sil.org. \vspace{-2mm}
%\bibitem{Palmer} Palmer, M., Gildea, D., Kingsbury, P. (2005). \emph{The Proposition Bank: An Annotated Corpus of Semantic Roles}. Computational Linguistics 31(1), 71-106. \vspace{-2mm}
%\bibitem{Eagles99} Sanfilippo, A. and et al (1999). \emph{EAGLES LE3-4244, Preliminary Recommendations on Lexical Semantic Encoding, Final Report}. \vspace{-2mm}
%\bibitem{sowa} Sowa, J.F. (2000). \emph{Knowledge representation: logical, philosophical and computational foundations}. Pacific Grove: Brooks/Cole. \vspace{-2mm}
%\bibitem{unl} Uchida, H. (2005). \emph{Universal Networking Language}. http://www.undl.org.
\end{thebibliography}
\end{document}

\noindent
{\small
{\bf Example} \\
Consider the sentence {\it ``John started to read at 10:45"}. The ISO-TimeML annotation structure for this sentence would be as follows, where we identify markables by the corresponding segments in the sentence and use self-explanatory names for particular elements of the conceptual inventory. \\
\ \\
event structure for start event: \\
$<``started", <process, past, inchoative, individual, positive>>$ \\
event structure for read event: \\
$<``read", <process, --, --, individual, positive>>$ \\
instant structure: \\
$<``10:45", <${\sc cet}, $<2008, --, -->$, 1045$>>$ \\
subordination structure: \\
$<<``started", <process, past, inchoative, individual, positive>>,  \\
<``read", <process, --, --, individual, positive>>,$ {{\sc initiates}$>$ \\
temporal anchoring structure: \\
$<<``started", <process, past, inchoative, individual, positive>>,
<``10:45", <${\sc cet}, $<2008, --,-->,  1045>>$, {\sc at}$>$ }\\


%\begin{wrapfigure}{l}{0.5\textwidth}
%    \centering
%        \includegraphics{hierarchy.eps}
%    \caption{\small{\emph{AGENT-roles hierarchy}}\label{hierarchy}}
%\end{wrapfigure}

