Inspiration : 36:40

It is not allowable for the sun to reach the moon, nor does the night overtake the day, but each, in an orbit, is swimming.

Isn't it amazing that everything in this world is actually swimming, when we think about it. Each event or action is related to the previous one and flows from it to the next. 

If I have to move a pen from my left side to my right side, it has to actually flow from the left side to my right as my hand picks it up and then drop it. It cant vanish from the left side and appear instantly on the right side. There is always a path that is followed and can be traced.

Isn't that amazing. I am reading a lot about how our mind understands and recognise environments and space that I realised the importance of 'swimming' in our life. Our mind forms maps of our environments and these maps are called cognitive maps.


Cognitive maps are highly imprecise maps and are nothing like the google maps. But its automatic for us and we use it to understand where we are and move to another location. 


These cognitive maps form a representation or a snapshot of a view of the environment and each such representation connects to the next to form a map.


 Its always the flow -> From Here to This to There

Like 1 -> 2 -> 3 -> 4 -> 5

Is Swimming...

So what does all these lead to. More questions ofcourse. Is it that the world is designed so that everything is swimming or is it our mind that perceives everything to be swimming. We have no way of knowing what the world is without our perceptions about it. Our perceptions are just perceptions that cant be always accurate, like the illusions that we perceive to be true.

Sun, moon , night and day are all swimming in orbits ( paths ). Revealed more than 1400 years ago.


 
 
Glory ! What is Glory! I have been trying to comprehend the meaning of Glory for some time now. Glory is not a common word that we use in our day to day conversations. However, its a common word in all religious scriptures - bible, torah , quran.  Its found in numerous forms in the Quran. Another word that closely resembles (lexically) Glory in quran is Yasbahoon (36:40) which means swimming or to glide.

From the discussion that i had on Glory, I learned that it means that God is Most Perfect.  We glorify that which is Most Perfect in all sense. Of course, God's creations - us and all that we see around us, are also perfect. But God is More perfect than that. 

In order to understand this, we discussed the example of an apple falling from an apple tree. When an apple falls to the earth, it follows the most perfect path to reach the earth. But we never glorify an apple fall. This is because, external influences - like the wind or a bird can cause the apple to deviate from the perfect path. God is Perfect in the sense that He is always Perfect and none can cause him to deviate from that state of Perfectness. Glory be to God.

17:44
The seven heavens and the earth and whatever is in them exalt Him. And there is not a thing except that it exalts [ Allah ] by His praise, but you do not understand their [way of] exalting. Indeed, He is ever Forbearing and Forgiving.

Allah knows Best. Here is a list of all words in the Quran that has the arabic alphabet 'seen' followed by 'ba' which is followed by 'ha' (س 12 ب 2 ح 6 ). These alphabets were chosen because Subahana has these alphabets and Yasbahoon has these alphabets. I am not clear how Yasbahoon is related to Subahana. Here is an attempt to understand them.

Please note that, this is not an attempt to explore hidden meanings in the Quran. However , this is an attempt to understand the Quran and Allah. The following list may be used to study the meanings of Glory.
VERSE WORD
2:30 نُسَبِّحُ
2:32 سُبْحَانَكَ
2:116 سُبْحَانَهُ
3:41 وَسَبِّحْ
3:191 سُبْحَانَكَ
4:171 سُبْحَانَهُ
5:116 سُبْحَانَكَ
6:100 سُبْحَانَهُ
7:143 سُبْحَانَكَ
7:206 وَيُسَبِّحُونَهُ
9:31 سُبْحَانَهُ
10:10 سُبْحَانَكَ
10:18 سُبْحَانَهُ
10:68 سُبْحَانَهُ
12:108 وَسُبْحَانَ
13:13 وَيُسَبِّحُ
15:98 فَسَبِّحْ
16:1 سُبْحَانَهُ
16:57 سُبْحَانَهُ
17:1 سُبْحَانَ
17:43 سُبْحَانَهُ
17:44 تُسَبِّحُ
17:44 يُسَبِّحُ
17:44 تَسْبِيحَهُمْ
17:93 سُبْحَانَ
17:108 سُبْحَانَ
19:11 سَبِّحُوا
19:35 سُبْحَانَهُ
20:33 نُسَبِّحَكَ
20:130 وَسَبِّحْ
20:130 فَسَبِّحْ
21:20 يُسَبِّحُونَ
21:22 فَسُبْحَانَ
21:26 سُبْحَانَهُ
21:33 يَسْبَحُونَ
21:79 يُسَبِّحْنَ
21:87 سُبْحَانَكَ
23:91 سُبْحَانَ
24:16 سُبْحَانَكَ
24:36 يُسَبِّحُ
24:41 يُسَبِّحُ
24:41 وَتَسْبِيحَهُ
25:18 سُبْحَانَكَ
25:58 وَسَبِّحْ
27:8 وَسُبْحَانَ
28:68 سُبْحَانَ
30:17 فَسُبْحَانَ
30:40 سُبْحَانَهُ
32:15 وَسَبَّحُوا
33:42 وَسَبِّحُوهُ
34:41 سُبْحَانَكَ
36:36 سُبْحَانَ
36:40 يَسْبَحُونَ
36:83 فَسُبْحَانَ
37:143 الْمُسَبِّحِينَ
37:159 سُبْحَانَ
37:166 الْمُسَبِّحُونَ
37:180 سُبْحَانَ
38:18 يُسَبِّحْنَ
39:4 سُبْحَانَهُ
39:67 سُبْحَانَهُ
39:75 يُسَبِّحُونَ
40:7 يُسَبِّحُونَ
40:55 وَسَبِّحْ
41:38 يُسَبِّحُونَ
42:5 يُسَبِّحُونَ
43:13 سُبْحَانَ
43:82 سُبْحَانَ
48:9 وَتُسَبِّحُوهُ
50:39 وَسَبِّحْ
50:40 فَسَبِّحْهُ
52:43 سُبْحَانَ
52:48 وَسَبِّحْ
52:49 فَسَبِّحْهُ
56:74 فَسَبِّحْ
56:96 فَسَبِّحْ
57:1 سَبَّحَ
59:1 سَبَّحَ
59:23 سُبْحَانَ
59:24 يُسَبِّحُ
61:1 سَبَّحَ
62:1 يُسَبِّحُ
64:1 يُسَبِّحُ
68:28 تُسَبِّحُونَ
68:29 سُبْحَانَ
69:52 فَسَبِّحْ
73:7 سَبْحًا
76:26 وَسَبِّحْهُ
79:3 وَالسَّابِحَاتِ
79:3 سَبْحًا
87:1 سَبِّحِ
110:3 فَسَبِّحْ
 
 
 
 
A God should be so Powerful that we shouldn't have to buy him or pay money to get to Him.
 
 
88:8  وُجُوهٌ يَوْمَئِذٍ نَّاعِمَةٌ
[Other] faces, that Day, will show pleasure.


75:22  وُجُوهٌ يَوْمَئِذٍ نَّاضِرَةٌ
[Some] faces, that Day, will be radiant,

80:38  وُجُوهٌ يَوْمَئِذٍ مُّسْفِرَةٌ
[Some] faces, that Day, will be bright -



 
 
AIM :-
Study whether 'Correlation Coefficients' (Pearson's and Spearman's)  can be used to find lexical similarity between ayats in the Holy Quran. Compare 'Correlation Coefficients' with Cosine similarity in finding in finding similar ayats from the Holy Quran


INTRODUCTION :- 
Pearson product moment correlation coefficient and spearman's rank correlation coefficient were used in the study. All the three - Pearson's coefficient, Spearman's coefficient and cosine similarity shows least similarity when value is close to 0 and show strongest similarity when value is close to 1 or -1.

http://en.wikipedia.org/wiki/Correlation_and_dependence
http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient
http://en.wikipedia.org/wiki/Spearman's_rank_correlation_coefficient

METHOD :-
For every two ayats/verses in the holy quran, compute the pearson coefficient, spearmans coefficient and cosine similarity. Compare them.

ABBREVIATIONS USED :-
1) pearson' product moment correlation coefficient - PEARSON_COEFF
2) spearman's rank correlation coefficient - SPEARMAN_COEFF
3) cosine similarity - COS

RESULTS AND OBSERVATIONS:- 
The web application for finding out the PEARSON_COEFF and SPEARMAN_COEFF between any two ayats of the Quran is here :- http://mobware4u.appspot.com/quran_correlation_coeff_analyzer

For cosine similarity :- http://mobware4u.appspot.com/quran_cosine_similarity_analyzer

In most of the cases, the correlation coefficients gave similar results as cosine similarity. i.e when cosine similarity value was high the correlation coefficients were also high and vice versa.

some examples :-
BETWEEN PEARSON_COEFF SPEARMAN_COEFF COS
88:10 & 69:22 1.0 1.0 0.9999999999999999
101:3 & 86:2 0.9878194944379475 0.9435173299101413 0.9945349461098567
101:7 & 69:21 1.0 0.9999999999999999 1.0
3:10 & 84:19 -0.12272854313719152 -0.12199044916180612 0.29410051324863773
3:10 & 56:64 -0.12345227218874748 -0.05533423060833531 0.39343980462128547
But at times, the correlation coefficients showed a lesser similarity than the cosine similarity analysis. 
An example
BETWEEN PEARSON_COEFF SPEARMAN_COEFF COS
112:4 & 51:9 0.6633358433414283 0.561769413436735 0.917793569237444
On examining the verses, it was found that the verses (112:4 and 51:9) were in fact very different verses. This shows that the correlation coefficients were better in finding the lexical similarity between verses than the cosine similarity.

There were two cases in which the correlation coefficients showed a greater value than the cosine similarity. They are :-
BETWEEN PEARSON_COEFF SPEARMAN_COEFF COS
88:875:22 0.9342171241831415 0.9922141844754531 0.984891553355433
82:381:6 0.9706153746658611 1.0 0.9853244887349389
The two cases ( 88:8 & 75:22, 82:3 & 81:6 ) had one word that was different.  This showed that Spearman's coefficient was incorrect in this case as it gave a very high similarity between the two different ayats.

Also, it as found that a Spearman's Coefficient of value greater than 0.9999999999999999 does not necessarily mean that the ayats were the same. The following example shows ayats which are not same but gives a spearman coefficient value of 1.
BETWEEN SPEARMAN_COEFF
42:2 & 26:1 1.0
82:381:6 1.0
For Pearson's coefficient, a value greater than or equal to 0.9999999999999999 always indicated the same verse.

For cosine similarity, a value greater than or equal to 0.9999999999999998 always indicated the same verse.

CONCLUSION :-

Correlation coefficients could also be used in the study of lexical similarity between ayats of the Holy Quran. Pearson's coefficient and Cosine Similarity were best suited to find the exact matching verse or repeated verses in the Quran. Otherwise, the correlation coefficients (both pearson's and spearman's) gave better results than cosine similarity.

 
 
Four different methods to compare ayats of the holy quran for lexical similarity were studied. The methods used were :-

  1. Verse verse string comparison
  2. cosine similarity comparison
  3. pearson's correlation coefficient comparison
  4. spearman's correlation coefficient comparison.

The above methods were used to compare ayats of the Holy Quran. The total time taken to repeat the methods for the entire quran was recorded. Each method was run 3 times for the entire Quran. Java was the programming language used. Here are the results :-
METHOD RUN 1 (ms) RUN 2 (ms) RUN 3 (ms) AVERAGE (ms)
STRING COMPARISON 600 550 524 558
COSINE SIMILARITY 56060 57743 55681 56494.67
PEARSON COEFF 135415 136801 135625 135947
SPEARMAN COEFF 990541 993030 1033356 1005642.33

The results clearly shows that string comparison in JAVA is faster than the other methods. However it has the following drawback which are not present in other methods :-

  1.  Arabic in quran uses extra characters in the verses to denote pauses. These characters are not part of the ayat itself. But can cause a string comparison to fail even if the ayats are the same.
  2. String comparison doesn't give a value that tells us how close two strings are. It can just be used to check if two ayats are the same or not.
 
 
Excellent resources to study the quran here :-
http://www.textminingthequran.com/wiki/Main_Page
 
 
Here is a tool to study the repeated verses or ayats that occur in the Holy Quran.

http://mobware4u.appspot.com/quran_repeated_verses

Some interesting Facts 

  1. The most repeated verse is : 55:13 (31 times) 
  2. Other repeated verses that are repeated more than five times : 77:15 (11 times), 26:9 (8), 26:108 (8) and 26:8 (6) 
  3. The surah that contains the most repeated verse is : Surah 026 - Shu-'araa' with 12 repeated verses 
  4. The surahs that have more than 5 verses repeated elsewhere are Surah 007 - A'-raaf (9), Surah 015 - Al-Hijr (11) and Surah 037 - Saaffaat (9)
  5. A total of 37 surahs out of 114 contain verses that have been repeated elsewhere.

10 MOST Repeated Ayats
  1. 55:13::: (31 times)  :::55:16,55:18,55:21,55:23,55:25,55:28,55:30,55:32,55:34, 55:36,55:38,55:40,55:42,55:45,55:47,55:49,55:51,55:53,55:55,55:57, 55:59,55:61,55:63,55:65,55:67,55:69,55:71,55:73,55:75,55:77
  2. 77:15::: (11 times) ::: 77:19,77:24,77:28,77:34,77:37,77:40,77:45,77:47,77:49,83:10
  3. 26:9::: (8 times) ::: 26:68,26:104,26:122,26:140,26:159,26:175,26:191
  4. 26:108::: (8 times) ::: 26:110,26:126,26:131,26:144,26:150,26:163,26:179
  5. 26:8::: (6 times) ::: 26:67,26:103,26:121,26:174,26:190
  6. 10:48::: (5 times) ::: 21:38,27:71,34:29,67:25
  7. 26:107::: (5 times) ::: 26:125,26:143,26:162,26:178
  8. 26:109::: (5 times) ::: 26:127,26:145,26:164,26:180
  9. 36:48::: (5 times) ::: 21:38,27:71,34:29,67:25
  10. 37:80::: (4 times) ::: 37:121,37:131,77:44

TOP 3 Surahs with the Most Repeated Ayats

  1. Surah 026 - Shu-'araa with 12 ayats
  2. Surah 015 - Al-Hijr with 11 ayats
  3. Surah 037 - Saaffaat with 8 ayats