Migrating from OthAuth to CakePHP Auth

This post lists down steps for migrating from OthAuth component to CakePHP’s inbuild Authentication component (Auth Component).

Note about using these migration steps: I used OthAuth mainly for authentication purposes i.e. mainly for verifying a username and password. I never got involved with complex authorization. So, these steps are only applicable if you are using Auth component withouth ACL.

1. In AppController, comment out all the code pertaining to OthAuth component and  add a beforeFilter function, as shown below


function beforeFilter()
{
   //Change hashing function to md5
   Security::setHash(’md5′);
   //Set password field as passwd
   $this->Auth->fields = array(’username’=>’username’, ‘password’=>’passwd’);
   //Explicit set session key so that we can AuthHelper (read below for more details)
   $this->Auth->sessionKey = ‘SomeRandomStringValueThatMakesSenseToYou’;
   //Set authSessionKey to be used by AuthHelper
   $this->set(’authSessionKey’, $this->Auth->sessionKey);
}


Few notes about the above beforeFilter() function

• By default, OthAuth component uses md5 encryption whereas CakePHP uses SHAH1 encryption. Currently, passwords in my database are md5 encrypted, hence, I am forcing CakePHP to use md5 by calling Security::setHash method.
• Also, note that CakePHP uses “Session.salt” value along with md5 encryption. OthAuth component had no such feature. Because of this, the encrypted password values as returned by OthAuth and Auth component, even if both the components are set to use md5 encryption, will differ. To make sure that the two password values match, change Session.salt value in app/config/core.php to an empty string.
• The last two lines are not necessary but are important to mimize changes in Views. They are required to use AuthHelper (see below)
• By default, CakePHP uses Auth.{$userModel} as the session key and, if not explicitly specified, it leaves the sessionKey variable as null. I guess this is a bug in Auth Component as it causes inconsistency in the logic. To overcome this inconsistency, I am explicitly setting sessionKey in line 3 and passing it to the view in line 4. Now, using this variable, AuthHelper can fetch user information.

2. In all other controllers (that are specific to your website), add following beforefunction()


function beforeFilter(){
     parent::beforeFilter();
     $this->Auth->allow = array(’list of function that should be allowed to non registered users’);
}


Also look at deny method. If you want to allow all the methods except one or two, then a better approach is to allow all and then deny only specific one. See the example code below


function beforeFilter(){
     parent::beforeFilter();
     $this->Auth->allow = array(’*') // * indicates allow all
     $this->Auth->deny = array(’add’, ‘edit’) //deny add and edit methods
}

3. OthAuth component comes with a handy helper. This helper allowed to check whether a session is a valid or not and also fetch user information. Based on similar lines, I created a AuthHelper. To use AuthHelper, create a new PHP file in app/views/helpers/ with “auth.php” as filename. Paste the below code in auth.php


/*
*This helper provide access to user parameters such as username, id, etc., of currently logged in user
*Send comments and feature request to ragrawal at gmail dot com
*@author - Ritesh Agrawal
*@version 0.1.0
*@license MIT
*/
class AuthHelper extends Helper {
           var $hashKey = null;
           var $initialized = false;
           var $helpers = array (’Session’);

           function init() {
                      if (!$this->initialized) {
                                 if (!isset ($this->view)) 
                                            $this->view = & ClassRegistry :: getObject(’view’);
                                 if (!empty ($this->view->viewVars['authSessionKey'])) 
                                            $this->hashKey = $this->view->viewVars['authSessionKey'];
                                 $this->initialized = true;
                      }
           }

           function sessionValid(){
                      $this->init();
                      return ($this->Session->check($this->hashKey));
           }

           // Get User Variables
           function user($key) {
                      $this->init();
                      // does session exists
                      if ($this->sessionValid()) {
                                 $user = $this->Session->read($this->hashKey);
                                 if (isset($user[$key])) 
                                            return $user[$key];
                      }
                      return false;
           }
}
?>

Notes about AuthHelper class: As you might have noticed, init() function requires a variable “authSessionKey”. We set this variable in beforeFunction() of AppController. It provides the session key name that Auth Component used to set user values.

4. Now, you can easily replace all instances of othAuth in all views by auth. More specifically, replace $othAuth->sessionValid with $auth->sessionValid and $othAuth->user with $auth->user, etc.

5. Final step. In AppController class add “Auth” component and “Auth” helper to the list of existing components and helpers.

References:
CakePHP cookbook has a well document section on Auth component.

 

<!–p

Microsft Word 2007 Bibliographic Feature - A Half Baked Idea

Within the scholar community (or alteast for me), there is much excitement with the introduction of new bibliographic manager feature in Microsoft Word 2007. But, yesterday as I struggled to keep up with a paper deadline, I was thinking that the bibliographic manager feature is more of a nuisance than a feature. Not that the idea of bibliographic manager is bad; it’s a great idea. But, its the lack of support and it’s limited capability that are bad. It seems that the idea of Bibliographic Manager in Word 2007 is currently half baked. Below are some of the issues that even a novice can easily identify with Microsoft’s bibliographic manager:

1. Add/Import Functionality -  There are two ways to add an article in microsoft’s bibliography manager. First, manually enter all the article metadata information by pressing thousands of keyboard keys. I feel laughing thinking how naive microsoft is. For eons, the scholar community has successfully used bibtex, RIS, and other standard metadata structure to share articles. However, microsoft word doesn’t offer any feature to directly import articles using these any of these standard formats. Luckily, I developed this Microsoft Word Plugin that allows directly import article from Memento, saving my finger from typing all the article details.

Second, import Microsoft bibliographic XML that someone else created. Agreed there are tools that can convert bibtex and other formats into Microsoft’s Bibliographic XML, but it means that to add an article you have to juggle data between tons of different softwares. In a time critical situation, which is common when trying to keep with a paper deadline, this seems a foolish idea.

2. References and Citations - The second most important feature that a bibliographic manager should provide is an ability to quickly cite article and build bibliography. Microsoft Word allows this. Great !!!. It also offer, by default, seven different ways to cite paper and list references. Great!!!. But wait - what happens if you are required to use some other format not available by default in word. hmm…Microsoft did think about it. They are using XSLT (XML Stylesheet) to format citations and references. Awesome! finally Microsoft learned to think ahead. But wait, don’t get too excited. The problem is even before you can create your custom XSLT, you will need to spend thousands of hours first trying to understand all the weird tags to format citation and references. I consider myself sufficient good in programming and also has an understanding of XSLT. However, even after spending hours, I wasn’t able to create a XSLT that I could have used to publish my bibliography in ACM Reference Format.

Then, think of digital immigrants (our old professors and people who avoid technology). Couldn’t microsoft have provided a simple GUI to quickly a custom XSLT. I can easily envision a simple user interface where a user can drag and drop different parameters and insert any extra text (such as [, ], “, etc). But, if Microsoft had thought about it before, then It would not have been Microsoft but Apple

3. Export Feature: How dare you even think about it :). Its microsoft and known for making thing worse (well not always )

The Verdit: Adding Bibliographic Manager in Word 2007 is a step in a right direction and using XSLT to format citation and references is a good idea, but there is a long way before it can be used effectively and efficiently. My final words will be, TRY IT BUT DON”T RELY ON IT.

Evaluate your writing with Flesch Reading Ease

Today, I completed writing first chapter of my PhD thesis and Microsoft Word indicates that the “Flesch Reading Ease” score for the first chapter is 30. In order to better understand what this score means, I looked at Wikipedia. According to Wikipedia, Flesch Reading Ease score of 0-30 indicates that writing can be best understood by a college graduate. In general, higher the Flesch Reading Easer score, easier is the writing to read and comprehend.

 

Although, I felt good about my Flesch Reading Ease score, I also felt the need to compare my writing with some other published article. After doing some random searches, I came across this interesting article - “Rating the major computing periodicals on readability”.  This article analyzed ten major periodicals and found that average Flesch Reading Ease score varies between 25 - 50. Considering this and that my first chapter is still in draft stage and still got a Flesch Reading Ease score of 30, I think, my writing is reasonable

 

Flesch Reading Ease score for this article: 65. ( = if you are 13 and above, you should be able to read this post)

Tips:

1. An Online Tool to calcuate different readability statistics: Style & Diction
2. Books on improving writing skills
            1. The Elements of Style: Okie Book. Talks about simple grammatical mistakes that often non-native (and sometime native) speakers of english make in the writing.
            2. Expectations: Teaching Writing from the Reader’s Perspective : Great Book. Teaches how to write from reader’s expectation. The only thing I can say about this book is that it teaches what design patterns are to a programmer.
           3. The Pyramid Principle : Logic in Writing and Thinking: Okie Book. Talks about how to organize ideas.

 

Garmin Forerunner (301) to Google Maps

Yesterday I did my first 10 Mile run. I was very excited about this and wanted to share with everyone. However, I couldn’t find single webservice or software that could have turned my GPS data from Garmin Forerunner 301 into a nice simple Google Map as show below.

 

Google Map showing route information of my 10 mile run. Marker numbers are indicating number of miles I completed by that marker

 

(Elevation Image from MayMyRun.com)

 After many trial-and-errors, I found different approaches on how to convert your data from your Garmin GPS into a Google Map. Below are two of these approaches:

Approach 1 (Garmin Training Center, Google Earth, Google Maps)

Step:

1. Make sure you have Garmin Training Center 3.4 or higher. Starting 3.4 version, you can select any particular run and view it in Google Earth. To do so, select a single run from running history and then click on view > view in google earth.
   
2. Once in Google Earth, your run will be listed under “places” block. Now select your run and save into a KML file. Right click on the run and select “save place as”. In the dialog box, select KML as file type.
   
3. Now, you can import the above generated KML file in Google Maps. First login into your Google account and then go to maps.google.com. Click on “my Maps” tab and then on “create new map”. In the new map panel, click on import button and select the above generated KML file. Upload the file and now all your markers are imported in Google Maps.
   

Issue: The KML file generated using Google Earth contains only marker information. So for instance, after running 10 miles, it had around 1000 markers. There are two problems with having so many marker points

1. Google Maps shows only 200 markers at most. So if you want to see all your markers, you have to click on “next” button (see at the bottom of your map information)
   
2. Unless, you are interested in elevation, these markers are meaningless and all you need is single line.
   

Approach 2 (MayMyRun, Google Maps)
Steps:

1. MapMyRun.com is a great resource for runners. It has many features such as training log, import GPS data, etc that makes it unique resource for runners. Once logged-in, you can go to “Map New Run” and click on Import > From Garmin GPS. Make sure you have downloaded Garmin Communicator plugin.
   
2. MayMyRun will automatically detect your Garmin Forerunner and import all the running history and will provide a drop down menu to select any particular run. Select the run that you want to map.
   
3. The great thing about using MayMyRun is that converts individual markers into single line and inserts only markers for each mile. Now from the map view, click on Save > “Save to Google Earth KML”. This will create a KML file that you can import in Google Maps
   
4. Follow step 3 of Approach 1 to import KML file into Google Maps.
   

Issue: You will need to create an account in MapMyRun.   Tips: Using Custom Icons: Instead of using default icons provided by Google Maps, I found these Glass Number Icons very useful. Numbers in the marker are indicating number of miles I completed by that marker.      

I ended up using Approach 2 as I already had an account with mapmyrun.com and it also solved the problem of multiple markers.   

Microsoft free goodies

Here are few sources to download free Microsoft software

1. Dreamspark- You can download Visual studio 2008, Windows Server 2003, Expression, SQL Server 2005, etc. The only requirement is that you should be a student.

2. SyncToy- SyncToy is a freely available Microsoft software for file synchronization. I have tried it and seems reasonable.

3. ActiveSync- syncrhonize files on your desktop with hand-held devices and vice-versa

4. Virtual PC - allows running multiple PC-based operating systems simultaneously on one workstation.

just found out..there is already a blog on free Microsoft products and it list more than 150 productions. Never knew Microsoft has so many software under its belt.

 

 

 

Rethinking Web Application Development Choices

Oh man…its has been more than a month that I blogged ..okie..so I am posting this half baked blog just to keep my blog rolling. I am still working on this post, trying to create a comparative table, but till then below are some thoughts on web application development choices.

A dynamic web application, minimally, involves 1) a database, 2) a server side scripting language, and 3) a front end. At each level, developers have many choices to choose the right combination of technology for their web application. While, it is good to have choices, one also has to carefull in picking the right technology depending on his own level of competency and project requirements. Based on my own experience, I have jotted down some choices and concerns one should be aware of when developing a web application:

Databases:
Choices:MySQL, Postgres, Oracle, DB2, Objectivity, MSSQL, etc.

When picking a database for any project, I always think about these concerns:

1.  RDBMS vs. ODBMS vs. ORDBMS -Relational Database Management Systems (RDBMS) are one of the most widely used database management systems (DBMS). For most applications, they provide robust solution. However, certain applications such as those concerned with hierarchy, RDF/OWL, object oriented can be more efficiently modeled using Object oriented databases (ODBMS) or object-relational database management systems (ORDBMS).
2. Open Source vs. Free vs. Proprietary -
3. XML Support -Some database such as MSSQL provide an inbuilt XML datatype. Thus, you can store, retrieve and query XML as any regular (int, text) datatype.  Although, I don’t like MSQL at all, I found this ability to be very useful in many applications. Mainly, as the web applications are moving towards Service-Oriented-Architecture (SOA), the ability of a database to natively handle XML gets increasingly useful.
4. GeoSpatial Support -Increasingly geospatial applications are becoming common. Google Mashup, Virtual earth, etc are just tip of the iceburg. Oracle has a long history of providing good support for geospatial applications. It supports basic topological operators such as overlap, intersection, etc. In open source community, increasingly, Postgres is becoming the default choice where-ever an application needs GeoSpatial support.
5. Full-Text Search -
6. Efficiency -
7. Deployment -
8. Community support -

Server Side Scripting Languages:
Choices:PHP, Ruby, Java Servlets, Python, Perl, etc
Concerns:

1. Community Support
2. Efficiency
3. Object Oriented
4. Scalibility

Front End:
Choices: On the front end side, the choices can be mainly divided in two types. First, the traditional technology of using HTML along with Javascript, CSS, and Ajax. Second, using rich internet application technology provided by Adobe (Flex) and Microsoft (Silverlight).

Concerns:

1. Search engine optimization - While traditional webpages can be easily scanned and searched by robots, this is not true for rich internet applications.
2. Application loading time  -if application loading time is critical for you, then using traditional HTML approach is a better idea. Flex and Silverlight requires loading libraries and there introduce significant latency when initially loading the application.
3. Rich interactive visualizations -If your application requires providing rich web based visualization such as interactive maps, graphs, network charts, etc., using RIA’s provided you with much more options.

 

 

Removing default values from CakePHP result

In most of my MySQL tables, I generally use some default values so as to make sure that the webpage doesn’t show up empty. However, as I was building a utility that allowed users to download their data, I wanted to remove these default values. So I wrote a small function, removeDefaults, that can remove all the default values from CakePHP query result. Its generic and has not hard coding. It uses create() function to get default values and use that to compare results. Below is the function and a demo on how you can call it.

// Insert this code in your in app_model.php

/**
* function: removeDefaults
* @description: Removes fields that contain default values in CakePHP result
* @param $array $results - Result array returned by CakePHP
* @param $array $ignore - Fields that must be ignored 
* return - $array 
*/
function removeDefaults($results, $ignore = array())
{
      $defaults = $this->create(); //load default values
      foreach($results as $key => $models){
          foreach($models[$this->name] as $field => $value){
              if(in_array($field, $ignore)) continue; //
              if($value == $defaults[$this->name][$field])
                     unset($results[$key][$this->name][$field]);
          }
      }
      return $results;
}


You can call the above function from any function available within your model. Here I will show I used it in afterFind
class sample extends AppModel{

       function afterFind($results){
            $ignore = array(”doctype”); // ignore the ”doctype” field. I want to have its default value
            return $this->removeDefaults($result, $ignore);
       }

}


Dummy’s guide for converting character set of a web application

When developing a web application, one has to be careful about the “character encoding”. There are many options such as latin1, utf8, etc, and the best character encoding depends on the language that your targeted audience use for writing. Since most  web applications are required to handle all types of languages and, thus, are required to handle all different types of characters such as í, é, á. The best way to handle all these various characters is to use UTF8 encoding.

Now the question is where to set UTF8 character encoding. And the short answer is “everywhere”:
          1) at the database level,
          2) at the interface/connection between database and client level (PHP), and
          3) at the data display level (HTML).

Below are the steps that I took to convert Memento from latin1 to UTF8 character encoding

(Note: Broadly these steps will remain same for converting any web application from one character encoding to another. But here I will assume that the application is being changed from latin1 to UTF8 encoding)

 1. Changing Database to UTF8: First step is to set database to UTF8 character set. Although MySQL has SQL commands for converting character set, based on my trials, I found that these commands are not full proof. What I ended up is adopting a script from maisonbisson to convert my database from latin1 to utf8. This script creates empty tables by copying the structure of the original tables and set their character set to UTF8. It finally, copies all the data from the original table to the new table, making necessary adjustments to char and text fields.

<?php
$username = “username”;
$password = “change_to_your_password”;
$hostname = “localhost”; 
$dbh = mysql_connect($hostname, $username, $password) or die(”Unable to connect to MySQL”);
@mysql_select_db(”your_database_name”,$dbh) or die(”Could not select first_test”);
$tables=mysql_query(’SHOW TABLES;’);
while($row = mysql_fetch_row($tables))
{
 $table = $row[0];
 //$changes = array();
 if(mysql_query(”CREATE TABLE conv_{$table} LIKE {$table};”) === false){
  echo “Failed creating conv_{$table}:”, mysql_error(), “\n”;
  continue;
 }
 if(mysql_query(”ALTER TABLE conv_$table DEFAULT CHARACTER SET=utf8;”) === false){
  echo “Failed Altering conv_{$table}:”, mysql_error(), “\n”;
  continue;
 }
 
 // copy the data into the new table, transcoding to UTF8
 $fields = mysql_query(”DESCRIBE {$table};”);
 $allfields = $selectfields = array();
 while($field = mysql_fetch_array($fields)){
  $allfields[] = $field['Field'];
  $selectfields[] = (stripos($field['Type'], ‘char’) !== false || stripos($field['Type'], ‘text’) !== false) ? “CONVERT({$field['Field']} USING utf8)” : “{$field['Field']}”;
 }
 
 if(mysql_query(”INSERT INTO conv_{$table} (”. implode($allfields, ‘, ‘) .”) SELECT “. implode($selectfields, ‘, ‘) .” FROM {$table};”) === false){
  echo “Failed Insert into conv_{$table}:”, mysql_error(), “\n”;
  continue; 
 }
 
 if(mysql_query(”RENAME TABLE {$table} TO preutf8_{$table};”) === false){
  echo “Failed rename {$table} to preutf8_{$table}:”, mysql_error(), “\n”;
  continue;
 }
 
 if(mysql_query(”RENAME TABLE conv_{$table} TO {$table};”) === false){
  echo “Failed rename conv_{$table} to {$table}:”, mysql_error(), “\n”;
  continue; 
 }
 //echo implode($changes, “;\n”); 
}
 

mysql_close($dbh);

?>

Step 2: Specifying Connection between MySQL and PHP to be UTF8: The second step is specifying UTF8 as the default character encoding for transferring data between database (MySQL) and server side code (PHP). To do so, use “SET Names” command (MySQL). According to MySQL Reference, SET NAMES indicates what character set the client will use to send SQL statements to the server (exactly what we wanted). Run this command each time your application establishes a new connection with database.

If you are using CakePHP, it can automatically set this for you. All you need to do is set ”encoding” variable in your database configuration file. Thanks to Dev weblog for posting this tip. Below is a sample database configuration file that I have used in my CakePHP application.

var $default = array(
              ’driver’ => ‘mysql’,
              ’persistent’ => false,
              ‘host’ => ‘localhost’,
              ‘login’ => ‘username’,
              ‘password’ => ‘my_password’,
              ‘database’ => ‘mydatabase’,
              ‘prefix’ => ”,
              ‘encoding’ => ‘utf8′
 );

Step 3: Setting Ouput (HTML) to be in UTF8: Final step is to specify the output to be in UTF8 character set. This is important otherwise browsers will have a hard time guessing the correct encoding. And the bad part is they are really bad in guessing the character set. As, in most cases the output is HTML and you can easily set character encoding of HTML by using the meta tag shown below:

                       <META http-equiv=”Content-Type” Content=”text/html; charset=utf-8″>

Just add this line in the “head” section of the html file. If you are using CakePHP, you can add this line in your layout.

That’s all.
Note: Depending how dirty is your data, you might need some more intermediary steps at the database level. In my case, just using the above script worked.

References:

Introduction to character code issues and related general readings
1. A tutorial on Character Code Issues: http://www.cs.tut.fi/~jkorpela/chars.html
2. Multilingual Forms - http://www.w3.org/International/questions/qa-forms-utf-8.en.php
3. Wikipedia on Character encoding - http://en.wikipedia.org/wiki/Character_encoding

Blogs on changing character code of database
1. http://www.oreillynet.com/onlamp/blog/2006/01/turning_mysql_data_in_latin1_t.html -
2. Script for converting character set: http://maisonbisson.com/blog/post/11972/freaking-mysql-character-set-encodings
3. Quick PHP Script to figure out bad UTF8 characters - http://mysqldba.blogspot.com/2006/08/quick-php-script-to-figure-out-bad.html

CakePHP Specific
1. CakePHP and Character Set in the database - http://nik.chankov.net/2007/10/01/cakephp-and-character-set-in-the-database/
2. UTF8 Multibyte Behaviour: http://bakery.cakephp.org/articles/view/utf8-multibyte-behavior

Think twice before publishing your blog

Today, without putting much thought, I published one of my blog post that I should have never made publicly available. Luckily, one of my colleaque realized the mistake I made and conveyed it to me. In less than two hours of posting my blog, I made the blog private again. I verified that no one is able to access that particular post. Not only, I logged out myself from my desktop to verify that the post is not publicly viewable, but also tried different machine to be extra cautious. I also searched on google for my blog and did find a link to it; but luckily google did not cache that page. I was happy that no damage is made.

However, this feeling of “no damage” didn’t last long. After few hours of making my blog private, as I opened Google Reader, to my horror, it showed one new post infront of my blog. Shakingly, I click on it and found that the blog post was already cached by google reader. Grrrr…….Even though, I made by blog post private, Google reader never bothered to check its status again and verify whether the cached version is currently public or private. Now, I can only hope no one else has my blog setup on any automated feed.

But this also tells me one more thing about RSS and associated technologies. They are good for sharing resources but have no provision for pulling resources back. Shouldn’t there be one for such cases?. I believe, in an ideal world situation, Google reader should have verified again that my blog post is public or not. However, this is not an ideal world. So just be careful and think twice, thrice and many more times before putting anything on your blog.

Google Scholar Webservice (Beta)

Google Scholar is a valuable resource for all students and academic professionals. However, its potential usage is seriously limited by the lack of an API. Since Google never bothered to create one, I tried to create one myself. I have created a REST service which can take a query string and returns back first 10 articles from the google scholar and the associated authors (that google display’s in the left column). Currently, the webservice will return article title, url, bibtex link, year, and cited by. Checkout the demo to get a better understanding.

I am still working on refining the webservice. Soon, I will adding options for
1. “number of results” - how many articles you want back (max will be 100)  Use “num” parameter to set number of results
2. “import format” - bibtex/endnote/ or any other format that google can provide Use “format” to get citation format  (1 for RefWorks, 2 for RefMan, 3 for EndNote, 4 for BibTex, and 5 for WenXianWang)

3. boolean option to tell the webservice whether you need authors or not.

Next version will also include authors and journal details of an article.

Currently, I have listed google scholar webservice as Beta. But if you are excited and want to use it, go ahead. I can’t gaurantee, how long it will work before Google decides to blocks my domain and my webserivce. But till then enjoy.

Let me know if you find this interesting or have any suggestions.

Enjoy.