arabiCorpus Instructions

General Information about arabiCorpus

What is a corpus?
- A corpus is simply a large collection of words from a given language. These words can come from a variety of sources.
- A person searches from a corpus by entering a search string, which is simply a string of letters or characters.
- Corpora (plural of corpus) are often used by linguists to learn about some phenomenon of language.
- The benefit of a corpus is that it contains real language that was spoken or written by speakers of that language.
- This allows the researcher the ability to see how the language is used in actual context instead of just in theoretical examples.
- arabiCorpus is a program that allows you to search from a large corpus of Arabic words.
- You can use the Tutorial and Instructions below to see the different uses and benefits of this program.
Important Things to Remember
- arabiCorpus allows you to search large, untagged Arabic corpora.
- arabiCorpus searches for EXACTLY the string you type in (and nothing else).
- arabiCorpus accepts most regular expression language in search strings.
- You can easily search for individual words and you have the ability to search for multiple words at once.
- You can search individual texts or combined corpora, which consist of multiple texts.
- Some filtering of the results is available after the search.
- Use the Tutorial below to help you with your initial searches and see the Instructions for additional details on using arabiCorpus.
Basic Corpus Information
- arabiCorpus has corpora in five main categories or genres: Newspapers, Modern Literature, Nonfiction, Egyptian Colloquial, and Premodern.
- You will select a specific corpus each time you perform a search in arabiCorpus.
- The names and word counts for the Basic Search corpora are found in this section. For a more detailed listing of the texts found in each corpus, see the 'Detailed Corpus Information' section.
- The total number of words of the whole corpus is: 173,600,000.

Detailed Corpus Information

Newspapers
- The bulk of the corpora come from newspapers and each newspaper corpus can be searched individually in Basic Search. The following list shows the years and locations for the current newspapers:
Premodern
- There are also various Premodern texts, two of which can be searched individually in Basic Search and others are available in combined corpora based on genre. Any individual text may be searched using Advanced Search. This list details the various corpora in this genre:
Modern Literature
- The next section of corpora comes from Modern Literature, primarily from novels. The countries represented with the novels are Egypt, Palestine, Algeria, Saudi Arabia, the Sudan, Syria, and Lebanon. Currently about half of the material in the Modern Literature corpus is from Egypt, about a fourth from Algeria, and lesser amounts from the rest of the countries. These are the novels found in the corpus as of Feb 2012:
Egyptian Colloquial
- There is a small combined corpus with some Egyptian Colloquial data.
Nonfiction
- The last group of corpora is in the Nonfiction category, which includes a large amount of material from an Islamic Discourse web-site along with some literary criticism, other scholarly (and not so scholarly) works, some political speeches, and some official UN and other diplomatic documents. These are the corpora found in this genre:
All
- The 'All' corpus consists of all the texts mentioned and has a total of 173,600,000 words.
- Remember that searching in the larger corpora such as 'All' or 'All Newspapers' will lead to much longer wait times.

Performing Searches in arabiCorpus

Basic Searching Instructions
1. Type a search word or phrase into one of the two boxes on the left.
2. Choose a POS filter for your search. If you don’t want to use a specific filter, choose ‘string.'
3. Select a corpus to search from the dropdown list of corpora.
4. Press 'submit.'
Detailed Searching Instructions
1. Enter a Search String
  - Type a word, phrase, or regular expression into one of the leftmost boxes (not both).
  - When entering a search string, a space means a space. If you type two or more words it will look for the exact phrase.
  - When searching for a noun, input the noun without the definite article (Al ال).
  - When searching for a verb, input the past and present huwa form of the verb separated by a comma.
  - To find all the examples of what you are looking for, you must be aware of Arabic spelling variations.
2. Select a Part of Speech Filter
  - Part of Speech Filters (Read this before reading the choices below)
    
    You must select a part of speech filter for every search you perform.
    
    Any part of speech filter (POS) other than ‘string’ will filter your results according to certain rules:
    
    The results will contain the search string alone, with only a space or punctuation before or after.
    
    The results will also contain the search string with the suffixes and/or prefixes that go with the chosen POS filter.
    
    All other instances of the string will be filtered out (those that are not the bare form or with the known suffixes and prefixes).
    
    Click on the different POS filters below to see how they will affect your search.
  - String
    
    The ‘string’ POS filter will not apply any filter to the results.
    
    EVERYTHING that contains your search string will be returned, no matter what is before or after that string.
    
    For example, if you type ktb كتب, you will get EVERY example of that string in the corpus, including the following:
    
    Alktb الكتب
    
    ktbtm كتبتم
    
    fyktb فيكتب
    
    AlmktbQ المكتبة
    
    ystktb يستكتب
    
    ktbrcAt كتبرعات
    
    Note that this does NOT mean it will find all examples with a particular root.
  - Noun
    
    Choose the ‘noun’ POS filter and the program will accept the bare search string you typed in along with the following attachments:
    
    The conjunctions wa- و and fa- فـ
    
    The attached prepositions ka- كـ and bi- بـ and li- لـ
    
    The definite article (ال)
    
    The pronoun endings (هن|كن|نا|هم|تم|تما|هما|ك|كي|ها|ه|ي)
    
    The alif tanwiin at the end marking indefinite accusative
    
    The dual ending markers
    
    For example, if you type in ktb كتب, it will accept:
    
    ktb كتب
    
    Alktb الكتب
    
    wAlktb والكتب
    
    llktb للكتب
    
    bAlktb بالكتب
    
    ktbh كتبه
    
    ktbnA كتبنا
    
    ktbhn كتبهن
    
    bktbhA بكتبها
    
    The program would also accept ktbAn if it found it, since it assumes you are typing in a singular, and it looks for possible dual endings added to it.
    
    But it will not accept (it will filter out) results that cannot possibly be nouns or that add more characters than a normal suffix/prefix, such as the following:
    
    ktbtm كتبتم
    
    fyktb فيكتب
    
    ystktb يستكتب
    
    ktbrcAt كتبرعات
    
    If you type a noun that ends with the feminine marker (Q ة), the program knows to allow forms that have a t ت instead when there are pronoun endings.
    
    For example, if you type ktAbQ كتابة it will accept:
    
    ktAbQ كتابة
    
    AlktAbQ الكتابة
    
    ktAbthA كتابتها
    
    bktAbtnA بكتابتنا
    
    The fact that you have filtered your results with the 'noun' POS filter does NOT mean that all your results will be nouns. It only means that, given the morphological ambiguity of Arabic, they COULD be nouns.
    
    For example, if you type in ktb كتب and choose ‘noun,’ some of the results will unambiguously be nouns, such as:
    
    Alktb الكتب
    
    bktbhA بكتبها
    
    while others will simply be ambiguous:
    
    ktb كتب (could be 'books' or 'he wrote')
    
    ktbnA كتبنا (could be 'our books' or 'we wrote')
    
    ktbh كتبه (could be 'his books' or 'he wrote it')
    
    As mentioned, it WILL filter out forms that are unambiguously NOT nouns, which can be very helpful.
    
    Choosing a filter, therefore, reduces the number of false hits, but does not eliminate them entirely.
    
    REMEMBER that this tool CANNOT and DOES NOT try to overcome the ambiguities of Arabic morphology. If a form is inherently ambiguous, you WILL find it even though, in context, it is NOT what you are looking for!
    
    Remember that the words in the corpora are not tagged for part of speech. So, in the end, YOU decide if the word is a noun, verb, etc.
    
    Note: Remember, it only searches for what you type in. If you type in a singular form, it will only search for that. If you want to find the plural forms, you must search for them separately or at the same time in a combined search.
  - Adj
    
    Choose the 'adj' POS filter and the program will accept the bare search string you typed in along with the following attachments:
    
    The conjunctions wa- و and fa- فـ
    
    The definite article (ال)
    
    But NOT the prepositions or the pronoun endings
    
    The alif tanwiin at the end marking indefinite accusative
    
    The masculine and feminine dual endings
    
    The feminine marker (Q ة) (Note: the 'noun' POS filter will NOT do this)
    
    Note that the program will only search for all forms if you input the masculine form, so do not add the feminine marker unless you ONLY want those results.
    
    For example, if you type in jmyl جميل, it will accept:
    
    jmyl جميل
    
    Aljmyl الجميل
    
    jmylQ جميلة
    
    wAljmylQ والجميلة
    
    But it will not accept (it will filter out) results that cannot possibly be adjectives, such as:
    
    jmylhA جميلها
    
    bAljmyl بالجميل
    
    If you want to try to find forms like jmylhA جميلها and bAljmyl بالجميل, you need to search for jmyl جميل as 'noun', not as 'adj'.
    
    Alternatively, you could search directly for jmylhA جميلها and bAljmyl بالجميل as strings, bypassing the POS filters.
    
    Remember that choosing 'noun' or 'adj' or any other POS filter does not mean that the results will actually BE that form or even that you WANT to find that form. It only means that you want it to allow that particular set of prefixes and suffixes through the search filter.
    
    As with nouns, if you want to find the plurals as well you must explicitly include them. It does not happen automatically.
  - Adv
    
    Choose the 'adv' POS filter and the program will accept the bare search string you typed in and the conjunctions wa- و and fa- فـ
    
    Nothing else will be accepted
    
    This category is handy for adverbs, but is also useful when you are searching for a specific form and not an entire conjugation.
    
    For example, if you want to find the noun ktb كتب only when it has the definite article preceded by b بـ (and not all the other possible forms of ktb كتب), type bAlktb بالكتب and choose 'adv'. It will accept the following:
    
    bAlktb بالكتب
    
    fbAlktb فبالكتب
    
    wbAlktb وبالكتب
    
    Nothing else will be accepted.
    
    The same technique works if you want to find a specific verb form (nqwl نقول) as opposed to all the conjugations of that verb.
    
    Remember, choosing the 'adv' POS filter does not necessarily mean you think the word is an adverb. It means that you want the filter to cut out everything except the specific form you typed in, plus that form with wa- و or fa- فـ.
    
    Choosing 'adv' is almost the opposite of choosing 'string'. 'String' accepts every occurrence of the string in the corpus, no matter what else surrounds it, and 'adv' accepts basically only what you typed in as a complete, isolated word and nothing else.
  - Verb
    
    Choose the 'verb' POS filter and the program will accept the bare search string you typed in along with the following attachments:
    
    The conjunctions wa- و and fa- فـ
    
    The particle li- لـ
    
    The future prefix sa- سـ
    
    The colloquial prefixes Ha- حـ and bi- بـ
    
    The pronoun endings
    
    The perfect and imperfect verb conjugation suffixes and prefixes
    
    IMPORTANT NOTE: The program assumes you will type in the masculine singular (huwa) form of both the past and the present tense, separated by a comma. However, if you only type the first, it will usually guess the second correctly.
    
    For example, if you type in ktb,yktb كتب،يكتب, it will accept:
    
    ktb كتب
    
    ktbt كتبت
    
    wktbwA وكتبوا
    
    ktbwhA كتبوها
    
    ktbth كتبته
    
    yktb يكتب
    
    fsyktbwn فسيكتبون
    
    lnktbhA لنكتبها
    
    It will also accept some fairly ludicrous forms like:
    
    flLktbny فلأكتبني
    
    nktbnA نكتبنا
    
    This happens because the program more or less mechanically applies the rules without asking what the resulting forms mean.
    
    The suffixes and prefixes are applied to the exact string(s) you type in and nothing else. (You will not necessarily find other forms of the verb).
    
    The program tries to handle double, hollow, defective, assimilated and hamzated verbs correctly, but its analyses are not perfect, so you should check the search strings it actually uses on the Summary Page.
    
    The program will usually guess when you put in one of these irregular forms but it may guess incorrectly. For more accurate searches of this type, see the 'Searching for verbs' section under 'Important Points on Searching.'
    
    The program will alert you when it has made a guess so that you will know to check which search strings it used.
3. Choose a Corpus
  - Select one of the corpora from the dropdown list labeled 'corpus.'
  - In addition to choosing what type of text you want to search, it is important to realize that choosing larger corpora, especially the corpus combinations, will often make your ‘wait’ time MUCH longer.
  - Therefore, if you are searching for an uncommon word, using the combinations can be valuable, since this will give you more examples.
  - On the other hand, if you are searching for a common word, using the combinations can be unhelpful; aside from taking a long time for the search, it will return so many results (possibly 10s of thousands) that you will not be able to deal with them efficiently.
  - You should be aware that EgyptChat includes some colloquial and a lot of fusha and mixed fusha and colloquial, while the literature and even the newspapers do include some colloquial.
  - For more information and a full listing of all the works that are found in each corpus, please see the 'Detailed Corpus Information' section.
Important Points on Searching
- Searching for verbs
  - There are two different ways to search for various verb forms in the corpora.
  - In Basic Search, the program does a large amount of guessing and figuring to create the actual search strings it uses to search the corpus.
  - In Advanced Search, you create the exact search strings the program uses. It doesn't try to second guess you. This gives you more control, but it also allows you to do searches that are relatively ridiculous.
  - The method for searching for verbs in Basic Search is similar to that of Advanced Search. The main difference is that Basic Search has only one verb POS filter whereas Advanced Search has multiple.
  - You can use the 'verb' POS filter in Basic Search to find a variety of verb forms. The program will usually guess which form you want. Consider the following possibilities:
  - Basic Search will always make guesses for what type of search you intended even if you do not input the search correctly.
  - The advantage to using Advanced Search is that you know exactly what the program will be searching for when you input a search string.
- Searching with or without vowels
  - The program automatically strips the vowels and kashidas from both the search string and the text before it searches.
  - In order to perform searches with vowels, you must use Advanced Search and check the box labeled 'include vowels.'
  - For more information on these types of searches, see the Advanced Search Instructions.
- Searching for words with hamzas
  - Words hamzas present a special challenge for all search engines because texts are incredibly inconsistent.
  - To help remedy part of this problem, the program automatically searches for all initial hamza possibilites (A ا and L أ and E إ and M آ) when any one of them is typed in.
  - The other issue is that texts can also be inconsistent with hamzas within words. This issue cannot be resolved automatically by the program, so you will have to account for the variability in your search.
- Searching for other spelling variations (alif maqsuuras, yaa's)
  - You must consider other spelling variations that may exist in the various texts found on the program.
  - For example, the Ahram 1999 corpus (which came from the internet) uses the letter yaa' for the alif maqsuura, although not consistently.
  - Likewise, most of the newspapers sometimes leave the dots off words that are supposed to be yaa's. So you must search for them also with [ey] if you want to get everything.
  - There are many other such variations you need to take into account to be able to find all of anything.
- Searching for more than one item at once
  - If you search for more than one item at once, it will sort the results all together.
  - This can be handy, for example, if you want all examples of a noun and its plural.
  - To search for two items at once, type in the items separated by a comma. Be sure NOT to add extra spaces.
  - If the words you type in go with different POS filters (say 'adv' and 'noun'), the results will be unpredictable.
  - It is not possible to search for more than one word at once when the 'verb' POS filter is chosen, since the comma separated list is used in verbs to list the various stems of the single verb.
- Searching for phrases
  - Typing a space will cause the program to look for a space.
  - Most of the POS filters make no sense when searching for phrases. However, sometimes they can be made to be helpful.
  - If you type: lA gbAr clyh لا غبار عليه and choose 'adv' it will allow:
- Searching with Regular Expressions
  - Using regular expressions can greatly increase the power of your search.
  - Some aspects of the regular expression language don't work yet, so be patient. Most parts do work however.
  - You can use the backslash characters that mean specific things:
  - You can also use quantifiers with these:
  - You can use square brackets to indicate a list of things, one of which can go there, and you can use quantifiers with the list:
  - You can use ^ at the beginning of this list to make it a list of things that cannot go there:
  - You can use ^ (outside of the brackets) and $ to indicate the beginning and end of a word, respectively:
  - You need to think carefully about how your regular expression is going to interact with the POS filter you have chosen. For example:
  - If you want complete control, choose 'string', and only what you cut out 'by hand' will be cut out.
  - Indicate alternation between whole forms using parentheses and the vertical bar. You must match your parentheses or you will get an error.
  - Using parentheses and the vertical bar in combination with square brackets and square brackets with the carrot (^) can be a VERY powerful way to design precise searches.
- Using Arabic Script in regular expressions
  - Regular expressions work in Arabic script as well as they do in English (\w = \و etc.)
  - However, most browsers display the backslashes and parentheses in bizarre and unpredictable ways. If you type the search in carefully, it should work, but if you try to read it after you have typed it in, or on the summary page, you will probably have trouble.
- Cutting out results by hand
  - Sometimes you know that the ambiguous morphology of a particular form is going to give you thousands of false hits that you don't want to look through to find what you are really looking for.
  - For example, if you search for all forms of the verb bne ybny بنى يبني, you will get many examples of the imperative form Abn ابن (almost all of which are really 'son') and of the third person feminine past tense form bnt بنت (many of which are really 'girl')
  - To cut out forms that otherwise would be found by a search, type the search, a space, one or two dashes, a space, and then a regular expression that indicates what you don't want
  - You can use a vertical bar to cut out multiple things.
  - Hopefully you get the idea. Like searching with regular expressions in general, this can be a very powerful tool in helping refine your searches.
- Searching for punctuation
  - This tool currently does not have a way to search for punctuation.

Analyzing the Results of a Search

Basic Information about Results
- The results come back after waiting for a few seconds.
- If you search for a very common word that produces tens of thousands of hits or more, the program will sometimes balk because the database it is using will limit what can be inserted.
- The results of a search are saved temporarily in a database, and you can access them by clicking on the words that appear in the red bar.
Summary Page
- This page comes up first automatically after a search, and you can return to it by clicking on 'summary' in the red bar.
- This page gives you summary information about your search:
- The latter bit of information is useful for comparative purposes. The corpora are of vastly different sizes, so the actual number per corpus may be misleading, but the number per 100,000 words can be compared.
- This page also displays various alerts according to the search terms you entered. Usually, these alerts will appear when you have used the 'verb' POS filter and will inform you of any guessing the program has performed and how you might need to change your search.
Citations Page
- Click on 'citations' in the red bar to see the 'hits' with the 10 words before and the 10 words after.
- By default, these citations are sorted by the word that appears directly before the word you searched for.
- If you would rather see the citations sorted by the word directly after the 'hit', click on the red sentence near the top of the page that says 'sort by word after.'
- The citations are shown 100 at a time.
- The far right column on the Citations page shows you the subsections of the corpus you searched.
- If you want to see more context than the 10 words before and the 10 words after, click on the number at the far left of the citation.
Subsections Page
- Click on 'subsections' in the red bar to see the totals for the various subsections of the corpus you searched.
- The results in this section are ordered from most frequent to less frequent in terms of number.
- The frequency per 100,000 for each subsection is also given.
- This page allows you to compare, say, Premodern Arabic with Modern, or to compare newspapers from various places.
Word Forms Page
- Click on 'word forms' in the red bar to see the exact forms that your search produced, orderd by frequency.
- Examining this list can give you important hints about normal usage.
- Examining this list can also help you see easily what kinds of false hits you are getting so you can work to eliminate them.
- It is strongly suggested that you examine this list for every search before you take the rest of the results seriously.
- Click on any word in the word form list and a window will open up with the citations for just that word form.
Words Before/After Page
- Click on 'words before/after' in the red bar to see a list of the common words directly before and directly after the 'hit' ordered by frequency.
- Examining these lists is a good way to scope out the main usages and collocations of the word you searched for.
- Examining the lists can also help you identify structures that you had not intended to search for that you want to cut out.
- If you want to see only the citations with that particular word before or after, click on a word in the list.
Collocates Page
- Click on 'collocates' in the red bar to see a list of the most common collocates that appear with your search term.
- You cannot click on the word to see the occurrences in context like with other results pages.
- Like the Words Before/After Page, this page allows you to see what words occur immediately before or after your search term, but it also allows you to see those words that commonly appear near your search term and not just right before or after.

Tutorial

First Search: Look for a single noun
- Click on the red Instructions above the submit button to get these instructions in a separate window (otherwise they will go away as you follow them).
- Perform a simple search for a noun.
- Analyze the results of your search.
  - Examine summary of search that appears, and notice how many examples it found, and how frequent these are in the corpus (per 100,000 words).
  - Click on 'citations' in the dark red bar.
  - Click on 'sort by word after' to sort the examples by the word after instead.
  - Click on 'subsections' in the dark red bar.
  - Click on 'word forms' in the dark red bar.
  - Click on 'words before/after' in the dark red bar.
  - Click on 'summary' in the dark red bar to go back to the summary page.
Try other single words
- Type jmyl into the leftmost box, choose the 'adj' POS filter, the Ahram 1999 corpus, and click submit.
- Type tqrybA into the leftmost box, choose the 'adv' POS filter and the Ahram 1999 corpus and run through the various options once you get the results.
- Type ktAbhm into the leftmost box, choose the 'adv' POS filter and the Ahram 1999 corpus, and look to see what you got under word forms.
- Type prb into the leftmost box, choose the 'verb' POS filter, the Ahram 1999 corpus, and run through the various options looking at the results. Note particularly the large number of forms under 'word forms'.
- Now run prb again, but this time choose the 'string' POS filter.
Account for variabilty in the corpus
- There are several ways to have the corpus look for alternates (either this OR that). The easiest involves square brackets.
- The square brackets tell the program to search for any one of the characters in the brackets.
- Type lbnAne into the leftmost box, choose the 'noun' POS filter, and the Ahram 1999 corpus.
- Now type msWwl into the leftmost box, choose the 'adj' POS filter and the Newspaper combined corpus. You will have to wait longer for the results because of the larger corpus.
- Now type tlfwn into the leftmost box, choose the 'noun' POS filter, and the Ahram 1999 corpus. Look how few the results are.
- Now type tlyfwn into the leftmost box, choose the 'noun' POS filter, and the Ahram 1999 corpus. Look how many the results are.
- The program is really just an automoton. It doesn't know what you want. It just looks for exactly what you type in. If you need it to pay attention to spelling variation, then you must tell it to do so.
- Moral: PAY ATTENTION to spelling variations in Arabic, and account for them in your searches, often by using square brackets or question marks.
Search for more than one word at once
- Type mktb,mkAtb into the leftmost box. Note that there is no space after the comma. Choose the 'noun' POS filter and the Ahram 1999 corpus.
- Now try the following combinations:
  - hjwm,hjmAt,hjwmAt,hjmQ,mhAjmQ (check out the word forms page to get an idea of relative frequency)
  - bTbycQ AlHAl,TbcA,bAlTbc (ditto)
  - hAtf,tlfwn,tlyfwn,mwbAyl,mHmwl,xlwy
Filter results 'by hand'
- Type lbnAn into the leftmost box, choose the 'noun' POS filter, and Ahram 1999 corpus.
- Let's say you want to look through all the citations with lbnAn, and you don't want to have to deal with all those lbnAny's.
- Try the following searches with and without the dash section and check the word forms page to see what it cuts out:
  - mktb -- ^m (this will cut out all word forms that begin with a miim, i.e. all that don't have some other prefix)
  - mktb -- h (this one will cut out all word forms that have a haa' in them, meaning that mktb with certain pronoun endings will be cut out.
  - mktb -- h|y|n (this one will cut out all word forms with a haa', kaaf, yaa', or nuun, meaning that maktb with all but second person pronoun endings will be filtered out. Note that the vertical bar represents alternation.
  - mktb -- h|y|n|km?A?$ (this one also cuts out the second person forms, but you have to be careful: if you just list kaaf by itself it will cut out everything, since kaaf is in the word itself. So you need the question marks and the dollar sign.
- A good strategy is to perform plain searches first, and then examine the word form list to see if there are any forms you would like to filter out by hand. If there are, create a 'dash' phrase to do the job.
- Note for those with a 'regular expression' background: You can consider the list after the dashes to be a regular expression for which the program automatically provides an outside set of grouping parentheses.
Search for a phrase
- Type lA gbAr into the leftmost box, choose the 'adv' POS filter, and Hayat 1997.
- Try these other phrases:
  - Type (Al)?bnyQ (Al)?tHtyQ into the leftmost box, choose the 'string' POS filter and Hayat 1997.
  - Type pyk bdwn rSyd into the leftmost box, choose the 'string' POS filter and Hayat 1997 (one hit only, check out the citation)
  - Type ttHlb lh Al[LA]fwAh into the leftmost box, choose the 'string' POS filter and All (one hit only, check out the citation)
  - Type \bwzArQ Al\w+ into the leftmost box, choose the 'string' POS filter and ShuruqColumns (to see a list of different ministries)
  - Try AlwzArQ Al\w+ (obviously not as useful)
  - Type mA lbV [LA]n into the leftmost box, choose the 'string' POS filter and ShuruqColumns.
  - Try (mA lbV|lm ylbV) [LA]n with the 'string' POS filter and Hayat 1997 (be prepared to wait)
- You are basically limited only by your imagination. However, if you want to be sure to get the results you want, remember to account for hamza variations and other spelling variations.
- Also remember that some writers do not put spaces where others do (some write lAbd and others lA bd). You can capture this with lA ?bd (try it with ShuruqColumns).
Compare results of different corpora
- Type hAtf into the leftmost box, choose the 'noun' POS filter and All Newspapers.
- Now type tly?fwn into the leftmost box, choose the 'noun' POS filter and All Newspapers.
- Type lqd into the leftmost box, choose the 'adv' POS filter and All.
- Type shl into the leftmost box, choose the 'adj' POS filter and Hayat 1997.
- Try the same thing with hjwm (with 'noun') and see if the subsection patterns make sense to you.

Miscellaneous Info

Recommended Browser
- You should be able to use this program with any of Google Chrome, Firefox, or Safari.
- Internet Explorer is the only browser that seems to cause some problems, so it is recommended to use one of the others mentioned.
Note About Numbers
- Because of how the numbers were originally entered into many of the texts making up the corpora, some of the numbers appear reversed or backwards after performing a search (i.e. 1999 may actually appear as 9991). We were unable to fix this problem, so you will simply have to accept that the digits of numbers might be reversed in any particular case.
- Therefore, we do not recommend that you use this program to search for numbers.
Warning
- This tool can easily become annoying if you are searching for a common word in a large corpus (you will have to wait a long time or the program may do nothing at all).
- When learning to use the tool, try a small corpus and search for slightly less common words.
Common Errors to Avoid
- Typing a noun or adjective with the definite article (the tool automatically looks for nouns and adjectives with and without the article; if you type it with the article, it will only find examples with it).
- Choosing the wrong POS filter (looking for an adjective with 'noun').
- Choosing 'noun' when looking for a phrase (you should usually choose 'adv' or 'string').
- Searching for a common word in the 'All' or 'All Newspapers' combined corpora.
Responses to Common Questions/Problems
- My results seem to be showing a mix of different searches. What is wrong?
  - You have probably just performed a search that took a very long time or appeared to time out.
  - The best way to manage this problem is to log out and wait a few minutes until performing another search.
  - It is also possible that you were logged in as 'guest' at the same time someone else was using the site as 'guest.' This can cause the same issue.
- Why do I want to filter my results?
  - If you are trying to find examples of a particular word or construction, and the program finds hundreds of results of something else that happens to match what you are looking for (because of the morphological ambiguity of Arabic), it can be very time consuming and annoying to search through the citations one by one looking for the ones you really want. If you can figure out a way to filter out the 'bad' ones, you can save yourself many hours.
- You want to take care of filtering yourself, and bypass the POS filters.
  - Choose 'string' and type a regular expression that matches what the POS filters do, or which varies them.
  - The program can deal with quite complex expressions efficiently, as long as parentheses are matched, so you can get quite a bit of control over what you are searching for if you want it.
- You want to find all examples of the verb twj, but looking at the results you realize that you have many instances of twjh, which could conceivably mean 'he crowned him' but which 99% of the time is in fact another verb: tawajjaha 'to head for'.
  - Choose 'verb' and search for twj -- h. This will delete all the twjh's. You may miss a couple of 'he crowned him's but it's worth it because you can now see that your results are mainly what you want and expected.
- You want to find all examples of the verb qAl, including passives.
  - In Advanced Search, searching for qAl,ql,qwl,ql with 'verb4' will give you all the actives. Searching for qyl,ql,qAl,ql will give you all the passives. Searching for q[Ay]l,ql,q[wA]l,ql will give you both all sorted together. Try this on a smaller corpus like 1001 Nights first, since it generates a huge number of hits.
- You want to find all examples of the verb Lyd ('to support).
  - In Basic Search, the program will often guess incorrectly when you search for this verb, thinking it is supposed to be a hollow verb.
  - In Advanced Search, searching for Lyd using 'verb' will miss all the imperfects and all the perfects where the hamza was not typed on the alif.
  - Searching for [LA]yd will get the hamza/alif variation, but will still miss the imperfects. To get those you need to choose 'verb2' instead and type: [LA]yd,Wyd i.e. the perfect stem and the imperfect stem with a comma in between. This should give you what you are looking for.
- You want to get comparative statistics on use of the verb forms yumkinu and tumkinu when they have a feminine verbal noun subject, but when you search for tmkn (using 'adv') you get overwhelmed with things that are really tamakkana, tamakkun, tumakkinu and the like.
  - This one is a good example of the inherent ambiguity of much of Arabic morphology, particularly graphemically. To get a statistic you can rely on, particularly if you don't have the stomach to actually read through hundreds of citations, you probably need to do what I call a 'limited search.' Instead of looking for all instances of tmkn, do a search for [yt]mkn \w+(Q|t(h|hA|k|hm|hmA|nA)), which will give you a list of all the words that end in a taa' marbuuta or a taa' followed by a pronoun ending after one of these verbs, and look at the list of words before and after. Look down the list of words after and pick out (say) the top 25 (or 10 or 5) feminine verbal nouns that come directly after ymkn or tmkn, and make sure they 'feel' right (i.e. you are pretty sure that the nouns you are choosing really are likely to function as the subjects of these verbs).
  - Then do a search for those forms only, once with ymkn before, and once with tmkn. You don't have an overall statistic, but you have a 'subset' statistic that you can trust. Sample search strings with 5 verbs I found would be:
  - Comparing the totals you get on those two searches should give you a good start at figuring out the relative frequency.

Announcements

13 August 2012: A new Nonfiction corpus of material from an Islamic Discourse web-site has been added.
2 July 2012: The newspapers mentioned below have been changed. Please see the Corpus Information for the updated word counts. The next section of AlGhad will appear on the corpus shortly. Let us know if there are any questions.
20 June 2012: The word count for some of the Newspaper corpora will change in the coming weeks. When we collected the texts originally, we were aware that there were some duplicate articles but we retained them because that is how they were archived on the Newspaper web-sites. We recently decided to remove as many of the duplicate articles as possible from these texts. If you have been using word counts for statistical research, your results may be altered slightly after the change. Therefore, we plan to wait to make the change until July 1, 2012. If you urgently need more time to finish research with the existing data or have any questions or concerns, please e-mail D. Parkinson (dil@byu.edu) immediately. Otherwise, we will proceed with the change as planned. The Newspapers that will be affected are Ahram 1999, Thawra, and AlGhad01. Another large section of AlGhad will also be added at that time.
22 February 2012: Over the next few weeks, arabiCorpus will be undergoing various changes. We are hoping to eliminate bugs and make the program more user-friendly. We will allow you to continue using the site as we make these changes, so be aware that things may change as you are performing searches. We apologize for the inconvenience but we are grateful for your patience as we try to make this program better for everyone.
20 February 2012: The list of all the texts found in the various corpora has been updated.
18 February 2012: Several premodern Arabic grammar texts have been added to the Premodern combined corpus and a Grammarians corpus has been created.
16 February 2012: A glitch causing an error in the frequency appearing on the Subsections page after performing a search has been resolved.
14 February 2012: Instructions have been edited and rewritten to reflect the changes that have taken place in the program. (This process is still ongoing).