Week 13 - End of the journey

Its the end of week 13 now and also the time for "Firm Pencils Down" date. 3 months have passed since I commenced work on this project. Its been a long and eventful summer and I got to learn a lot from.


The last week has been quite hectic with a flurry of pull requests to do bug fixes of existing bugs and the bugs which surfaced up recently and also to do changes according to feedback received by mentors. Here is a list of tasks I completed in the last week -


1. Fixed issue #451 - Now the modify page prompts users if they try to leave the page without saving the changes, in all themes. Also it gives the prompt only when the user has made a change.

2. Fixed issue #454 - Fixed the Global Index to show the alphabetical filters to sort the items

3. Finally fixed quicklinks and their tooltips - Fixed the overflowing quicklinks text in the basic theme. Also fixed the quicklink tooltips which now show both wikititle and full url to the item instead of just the wikititle.

4. Added css for smileys in basic theme which was missing earlier - Commit

5. Replaced important hex color values with variables in basic theme's theme.less. We should stick to using variables as much as possible as it makes it easier to understand and use and also make changes quickly - Commit

6. Fixed the invisible links in meta view in basic theme - The itemlinks in the meta section in the basic theme had the same color as the background color and hence became invisible. Changed the color of the links in this patch to fix the issue. - Commit

7. Added capability to edit acl string in item acl report view itself - Commit

8. Fixed the overflow of item names in the index view - Long item names used to break the css and overflow into a second line - Commit

9. Removed full stops from view titles - Commit

10. Fixed erroneous input element css in basic theme - There was unnecessary shortening of input textboxes - Commit

11. Fixed overlapping links in orphan view - The links did not have enough vspace between them. Added css to fix that - Commit

12. Right aligned time column in modernized theme history view - Commit

13. Made default acl string green in item acl report and made submit acl button smaller to make it look neater - Commit

14. Increased font size in global tags view in Modernized theme - The tag cloud in this theme earlier had very tiny font which was very hard to read - Commit

15. Removed extra commas and spaces between links in the meta view in basic theme - Commit

16. Made section heading size uniform across all sections in basic theme - Commit

17. Shifted Login button to the right in modify view in basic theme - Commit

18. Added title to icons in global history view and added padding under date titles - Commit

19. Fixed faulty footer which used to rise to the top at low resolutions in basic theme - Commit

20. Fixed failing modify view - The modify view was not being able to make changes to an item due to not being able to retrieve the Item ID - Commit


Phew!! Finally this has come to an end. It was a whole new experience and a whole new level of exposure to real-time, real-world work. I would like to thank both my mentors - Roger and Thomas for helping me and guiding me throughout the project and more importantly for giving me an opportunity to work on such a big project. Also I would also like to thank PSF and also Google because without them this would not been as good as it was. I hope to keep contributing to open-source projects. Cheers.

Week 8, 9, 10, 11, 12 Work

Ok guess it has been quite long since my last post.

Week 8

Just like week 7 week 8 was also about ACLs. This week I had to work on the Group ACL visualisation. In order to do that I had to first create a view to list all the groups present in the wiki just like the "Userbrowser" view that is already present in the admin section. We discussed the design of the view on the etherpad and we came up with this view -

The Group View

The names of the groups, their member users and groups are mentioned along with the links to the ACL reports for each of the groups. The list is sortable and is by default sorted by the group names and also the names of group members are sorted by default. Her is the commit for that. After this came the making of the Group ACL Report. For each group we decided to show only those items in the ACL reports which specifically mention the Group name in their ACL string. Here is what the view looks like -

The Group ACL Report

It lists the Item names or Item IDs in case of a nameless item. The names/IDs are hyperlinks to the modify view of the items so that the admin can directly go and modify the permissions of the item. Here is the commit for that.

I also worked on and committed the old patches - #445, Userwise ACL Reports, Metadata View in basic theme, Item ACL Report View.

Week 9

In this week I had to work on providing the capability to edit the ACL string for an item in the Item ACL Report so that the admin does not have to go to the modify item view of each item in order to change the ACL rights. So I created an extra text field in the Item ACL Report itself which contains the current ACL string for an item and the admin can modify the ACL there itself. The "default" tag specifies that the string for that item is same as the default ACL in the configuration. The codereview for that is here. Here is what it looks like -

The new Item ACL report with editing functionality

Also this week I had to work on removing the auto-computed metadata from the modify meta view. There was no use in giving them in the edit view as it would either have no effect or would give an error in case the user deliberately tried to change it. Here is the commit.

Week 10 and 11

This week I had improve UI elements and color schemes. Firstly I had to fix a few existing bugs in the UI. The first one was the #394 where there was an ugly mouseover and a validation error in the modernized theme. The namespace shown along with the location line used to show an ugly dropdown menu on mouseover and also it gave a validation error as a span element is nor allowed to contain a ul element. So we did away the showing the namespace over there. Instead we made a "Namespace" section in the "User" view itself. Hence it solved both problems together. Here is the commit for that. Here is how it looks like -

The Namespaces section in User view

Then came the #425 where the basic theme used to give a 404 not found error for common.css in the background as there was no common.css in the basic theme setup. So in order to fix this we decided to keep a uniform naming system for the stylesheets in all the themes. So I changed the names of basic.less to theme.less and basic.css to theme.css in basic theme and main.styl to theme.styl and common.css to theme.css in modernized and foobar themes. Here is the commit for that.

Also in one of the meetings Thomas pointed out that the URL for the User ACL Report is untidy due to unnecessary values being passed in the GET request. I removed those values to make it cleaner. Also he asked me reorder the options in the Admin Menu so that they are properly organised. Here is the commit regarding both of those issues.

Since I had a lot of time left in week 10 I decided to jump ahead and proceed to week 11 work, in week 10 itself. The first thing to do there was to add a background color to the sidebar in the basic theme so that it would look different from the content. So I made up a few samples and we finally decided that shades of blue would be the best choice among all as it would match the theme too. Here is the commit and the screenshot -

The new blue Sidebar

Then the Itemsize view and the Interwiki names view both needed a proper tabular view as they were rather clumsily arranged. Also the item sizes in the Itemsize view were not human readable and were rather the size of the items in bytes. I added bootstrap tables to both the views and also made the item sizes human readable. Here is the commit for them. The old Interwiki and Itemsize views were like -

And the new ones are here -

Also then I added CSS classes for making 2, 3 and 4 column lists. Here is the commit for them. Here is one sample usage -

Old single column list

New multi-column list

Next was to address the issue #66 in sharky93's repo which was about reloading the page when a new theme is selected so that the new theme gets automatically loaded. I added a line fo javascript in common.js -  "location.reload(true)" at the end of the form submission and processing so that current location is reloaded. The "true" argument is to force the page to reload from the server rather than from the cache. Here is the commit for that.

Also I made the Textarea in the modify form to expand/contract as the page is expanded or contracted, similiar to the way it happens in the modernized and foobar themes by making the width of the textarea as 100%. Here is the commit for that.

Last but not the least I removed the subitems sidebar in the modernized theme as it was not working as intended and removing that would have made it consistent with the other themes. Here is the commit for that.

Week 12

In this week I was supposed to add the capability for users to add their own custom css to the themes using wiki items as the source in the "User CSS URL" in the "Appearance" settings. We had planned to use a "raw" view to render the raw css as use that as the source. But Thomas pointed out that this capability would cause potential security threats as the users could make calls to malicious javascript codes using the url tag in the css. So this plan was dropped. So we planned on working on whatever is currently there and to improve it. Also the plan was to find flaws if any in the work done by me till now by practically using the wiki.

So first up this week was to finish open issues in my own repo and to report and solve new findings. The first one was to add CSS to the ACL reports in the foobar and modernized themes. I added the "zebra" class to the tables in both and modified a bit of css in order to make them look better. Here is the commit and the screenshot -

A small fix was to add an h1 header to the global index view in the basic theme to make it consistent with the other views. Here is the commit and screenshot -

A bigger task this week was to improve the global index view in the basic theme as it was poorly organized. I added bootstrap icons and tables to improve the view. Here are the commit and screenshots -

The old Global History view

The new Global History view

I also revisited the overflowing quicklinks problem. I now removed the delete quicklink "X" icon and instead gave the user the same option in the User Actions tab, as suggested by Thomas. Also since the icon is removed I increased the maximum characters in a quicklink to 20 as there was more space. Furthermore the tooltip now shows as "<wikiname> : <full url>". Here is the codereview for the changes. 

The next week would involve finishing the project after finding and fixing whatever bugs I can find. Hope that all goes well :D

Week 7 Work - ACLs

This week the first task was to find and solve bugs related to Access Control Lists referred to as ACLs hereafter. The access control list in simple metaphorical terms would be equivalent to a guest list at a big party. If your name is on the invited guests' list then you may attend the party else you stay out of it :D . So the access control lists basically specify what permissions a user has w.r.t to a particular object. A user may read, write, create, destroy, admin an item depending on the ACL corresponding to that item.

The first one was an already reported bug - #417 whereby the the userbrowser view in the admin section used to crash in case there were any users who had made accounts but not confirmed them via the emails sent to their accounts. This is caused as when a user registers on the wiki, their email address was stored under the "EMAIL_UNVALIDATED" key till they do not confirm the account. Once they confirm it, the email is stored under "EMAIL" key. So the earlier code used to only look for the email under "EMAIL" key while retrieving user info and as such crashed. I changed the code to look first for "EMAIL" and if that does not exist then look for "EMAIL_UNVALIDATED". Here is the commit for that change. Now you might think that where did ACL figure in all this. This is related to ACLs as one of the new ACL views I created this week would have its link from the userbrowser view where all the users would be listed. I'll talk more about that later.

The second bug that I came across was #444 whereby if the user tried to access something which they are not allowed to read (like gatecrashing that party I mentioned ;) ), they would see an error as mentioned in the bug report instead of a proper "Access Denied" page. This happens because somehow the registered 403 error handler does not spring into action in case a 403(AccessDenied) error is raised in this case. I am still working on this issue, you can find the related code reviews here.

Another bug was #445 whereby if a user changed the ACL of an item such that they were no more allowed to read that object, they would receive the error reported in the bug ( I wonder who'd strike their own name off the guests' list :D ). Basically the self.revs object being used there gets its value from the "get_items_last_revisions()" function. This funtion fails to find the item in case the user is not authorized to read it, and hence returned an empty list. This empty list caused the error. Hence I put in a check for empty lists and displayed an AccessDenied Error in such a case using the abort(403) function. The code review is here.

Yet another bug was #446 wherein a user could modify the ACLs of a particular object even if he had just the read and write permission, whereas according to the design only a person with admin rights should be able to do that. In the basic theme this error was caused because there was no check on the user's ACL capabilities before displaying the modify form, or in metaphorical terms there was a guest list defined but there was no one to cross check the people entering the hall. So I added a check for the user, and only if they have admin rights can they access the Modify ACL form. Here is the commit regarding that.

Finally the last bug I worked on was #447 which again like #417 was not directly involved in ACLs but was related to the view I needed for further ACL views. Here the previous developer had not kept in mind that the rev.meta[NAME] attribute would be a list as items can have multiple names and as such it could not be used as a key for a dictionary. So I unrolled the previous complex list comprehension and used for loops instead to take care of the list. Here is the commit regarding that.

Now comes the new views I worked for easy visualization of ACLs. The first one was the User ACL Info view. This view contains the information about ACL capabilities of the user with respect to each item in the wiki. This view can only be accessed by the superuser(s) specified in the wiki configuration through the userbrowser view, in the admin section, where there would be a list containing various information about each user (Ah! Now you see why I solved #417 and #447). The clicking on the item names in the list would take the superuser to the Modify Item page where they could modify the ACL of that item as they liked ( Thats if they had admin rights for that item as per #446 ). The tablesorter js library was used to make an easily sortable table. Here is the code review for that.

The User ACL Info view

Another view that I made was the Item ACL Report. This again can be accessed only by the superuser. Here the superuser will see a list of all items in the wiki and the corresponding  ACLs for each item in the wiki. Also clicking on the item name would take the superuser to the Modify Item view. Tablesorter js was used here too. Also in both new views if the item was nameless item, the Item Id would be shown instead. Here is the code review for that.

Item ACL Report view

In the continuing couple of weeks I would be working on the show Groups view and Group ACL Reports view to enable the superuser to visualize the ACLs and to easily change them.

Week 5 & 6 work

This week I started work on the meta-data section of the UI. Earlier the meta-data section of the UI was displayed as raw data. Below is the screenshot -

 

The earlier raw metadata view

As you can see the meta-data section just shows a dictionary with HTML Markup. In this code the item metadata was getting converted to a python dictionary and then to text using json.dumps() in the function "meta_dict_to_text()". It was then HTML escaped and put in a <pre> tag by the function named "_render_meta" in "Moinmoin/items/__init__.py". This was then returned to the "show_item_meta" function in the views section (Moinmoin/apps/frontend/views.py). This function would carry out the HTML Markup of the input using the flask Markup() function. This was passed on to the template and displayed as such.

I first made another function "meta_to_dict()"to just convert the metadata to a python dictionary so that I could use that dictionary in the template to extract data and customize it display easily. I further removed the duplicate part of the function in "meta_dict_to_text()" and instead called "meta_to_dict()" for the necessary processing. I also changed the name of the function from "_render_meta()" to "_meta_info" as it was no longer rendering info rather returning a dictionary. Also changed the variable "meta_rendered" to "meta", again because it was not a rendered data, rather a dictionary. In the template I showed the user friendly content-type along with the detailed one present earlier using the function "shorten_ctype()" added in the search UI patch. Also I added hyperlinks and visualisation for the validity of the links. If a particular item pointed by a link in "Item Links" list or "Item Transclusions" list does exist, the link will appear blue in color else it will appear red in color. Also I changed to "Modified Time" to user readable format. Same with the "Item Size". Here is how it looks like with my patch - https://codereview.appspot.com/110210043/

 

 The new metadata view

Also I worked upon fixing the footer to the bottom of the page which earlier used to come up to the middle of the page in case of a small content. Here is an example -

The footer midway up the page

I added the classes "moin-header", "moin-content", "moin-footer" to the layout.html template in basic theme in order for easy CSS selection. In the css I changed html, body elements to have a size of 100%. Also the container section to have a min-height of 100% and relative position. The moin-content was changed to have bottom padding of 100px in accordance with the size of the footer. The moin-footer also now has bottom as 0, position as absolute, and width of 100%. Here is how it looks now - 

The new footer at the bottom

CRs for this week :

1. Metadata

2. Footer

 

Also this week I finally got to commit the previous CRs-

* Added pygments classes - https://bitbucket.org/ajgupta93/moin-2.0/commits/82ffb3f45eed569ae2aeab92e3190bb244fac371

* Fixed footer to the bottom of the page - https://bitbucket.org/ajgupta93/moin-2.0/commits/4385775cf3a1aaaad070501c8f6e49ad03f11edd

* Improved Search GUI - https://bitbucket.org/ajgupta93/moin-2.0/commits/2003e3685733eb19c3052c19d419e36445caea64

* Added common.css and removed unnecessary pygments_default.css files - https://bitbucket.org/ajgupta93/moin-2.0/commits/9645576dd44d90f608cca417288a9f4d5fc63470

* Improved Index GUI - https://bitbucket.org/ajgupta93/moin-2.0/commits/011a118feecfe0037d0308257f19f332321d9053

I also realised that I should be making small patches so that they are easier to review and reviewed faster and as such committed faster. Due to large size of my previous patches they took too long to review and thus late commits.

Week 4 work - Index GUI and Pygments CSS classes

Currently the Index section in the Basic theme is completely broken. The options are in the form of an unordered list, making it hard for even to believe at first look that they were actually links to the actions and not just plain list items. The list of items is also quite confusing and does not have checkboxes, so the user can select a particular item to perform a specific action on it like "Download", "Delete", "Destroy". Also the section that should come up as a popup when a user tries make a new item is already present on the page and is completely jumbled up. The buttons are really in a mess. Here is what it looks like :

The Index UI top with the list of actions

Index UI bottom with all the messy buttons

The Index UI with my patch (Content Section is collapsed)

The Content Section expanded 

I converted the actions section from a list to a bootstrap navbar to make it easy for the user to understand the actions being carried out. Also now we have checkboxes for each selected item and as such the user can know which item they selected. I tried to organize all the options as cleanly as possible. Also now the New Item popup acts as a popup when "New Item" option is clicked, instead of constantly being present on the page.

The New Item popup

The Basic theme also was earlier not showing smileys even though they are defined in the markup, due to missing smiley images. I added the smileys to the basic theme static files so now the basic theme shows smileys too :D

Also to add was the css definitions for the pygments classes used to highlight various items like python code, etc in both the "Show Item" and "Syntax Highlighter" section. The wiki was not being able to highlight items due to lack of css class definitions even though the html for both sections used the css classes. I had added the definitions manually to the code, but Roger pointed out very well that it should be done at the time of install by the python code itself.

The code review for the above works can be found here :

1. Index GUI

2. Syntax Highlighting

Week 2 and Week 3 Work - Improved the search GUI and added Search Options

At the start of the project the MoinMoin search GUI was highly unorganized and lacked information. Here is how it looked like initially -

The initial MoinMoin Search

MoinMoin search now

Firstly I added metadata info to the search results. I added the "Item Size" and "Content Type" fields in the metadata. The "contenttype" field actually stored in the database is not so user friendly, it stores values like "text/x.moin.creole;charset=utf-8", which is not so easy to understand for a normal user. So I used a dictionary to display the corresponding user-friendly content-type in the results. We had also considered adding the "ACL Rights" but that was eventually deemed to be of not much use so we did not go ahead with that. Also added here is the color coding - Yellow for search suggestions, Green for metadata, Blue for link to the search result. Also earlier the search field in the navbar on the top used to be present in the search results page along with the AJAX search field that is present on the page. I removed it by adding

{% set search_form = None %}

in "MoinMoin/templates/search.html" which is basically the base template for the search page. So search form on the navbar is made when the search_form variable is set, here the search form does not get added to the page. Also the "Login" button used to remain in its place instead of shifting to extreme right when the search form was not present. We fixed that by adding "navbar-right" bootstrap class to the button. Then you might notice the re-positioning of the suggestion terms and query time. We thought that the search terms are more relevant to the search so they must be present near to the search bar. The query time on the other hand is more relevant to the results. Also since this is a AJAX search which searches as we type we thought that the user might get confused as to how to search (earlier there was no "Search" button) so we added a "Search" button.

Finally I added the "Search Options" section to make it easy for the user to search items. The options are present in an expandable div which opens up on clicking on the "Search Options" bar. There are three types of options - 

1. Revisions - If the user selects "Latest" option, then all the searches will be carried out on the latest revisions of the items. If they select "All", the search would be carried out on all revisions of all items. By default all searches are carried out on latest revisions.

2. Sort By Time - If the user selects "Older First" the results would contain items in an order such that the items modified earlier would be present closer to the top and those modified recently would be present closer to the bottom. By default the search results are not sorted at all. This is was carried out by using a facet in the whoosh api and using it in the "sorted_by" argument in the whoosh search query.

3. Content Types - The user might want to search for an item of a particular content type or a group of them, so they can select any type they like and the search result would contain items of that type only. By default the search contains items of all types. This was carried out by the using a filter for each content type the user selected.

Also one other thing we started this week was to put all the common css class definitions in a common css file instead of triplicating it in each of the three theme css files. So the common css for the search GUI was put into the MoinMoin/static/css/common.css from where each theme picks up the definitions.

The code review for the above changes is here.

GSoC Update - Week 1 Work

Just a week to go before the coding period started and I, out of nowhere, contracted typhoid and got hospitalized for 8 days. Luckily not much damage was done and just a couple of days into the week, I was discharged from the hospital. I had already informed my mentor Roger Haase about the delay in work and he was understanding enough to give me more time to start work and asked me to buck up for work ahead. Finally I looked at the plan for the week and started work.

The first issue in light was the issue #73. The sidebar was becoming too large in the Basic theme. The reason being the padding in the cells - the Home button, History, Index, User Actions, etc was 10px, which is quite high. It seemed as though one could fit the text three times in the same button :) . After reducing the padding the sidebar got smaller and looked better fitted. Another thing I did was to make the User Actions, View Options and Item Actions menu expandable. This made the sidebar much smaller and cleaner. The user can click on the '+' button to expand or collapse the menu. Here you can see the code review and commit.

A screenshot of the changed sidebar

The second issue pointed out to me was that the word settings was getting repeated too many times in the 'User Settings' menu in the menu bar. This caused problems in translated versions as for example in German the word 'Settings' translates to 'Einstellungen' which is quite long. As a result the menu bar breaks down and turns multiline. So I removed unnecessary repetition. Here you can find the code review and pull request.

Finally the main issue in focus - The Quicklinks in the Basic Theme. The quicklinks as I had mentioned in an earlier blog were exceeding their divs and as a result the remove button 'X' was also not working. I had also mentioned that it was being caused due to the quicklinks being only shortened to the default size of 25 characters whereas for divs in the Basic theme the ideal maximum size of a quicklink would be 15 characters. So I changed the code in the function call to the shortening function - "shorten_item_name" and passed in 15 as argument. Now another problem that was there was that the quicklink tooltip, which comes up on a mouseover, only showed the current wikiname and nothing about the item itself. So if I have a wiki named "MyMoinmoin", all quicklink's tooltips showed "MyMoinmoin" on a mouseover. We had a discussion regarding the solution to this in the weekly meeting on Saturday. We came to the conclusion that the format "<wikiname> : <itemname>" would be good enough as of now. So I changed the layout.html for basic and modernized themes to reflect the same. It is currently up for review. Here is how it looks like now.