YAML IPython notebook¶ Little experiment base on the fact that apparently YAML is made to be better readable by Humans than JSON. We've also had some complaint that metadata are not keep in nbconvert when roundtripping through markdown, those two made me think that I could try to see what ipynb files stored as YAML would look like. I'll also use this post to do some experiment for nbviewer future nbviewer features, if you see anything wrong with the css on some device, please tell me. First atempt¶ Apparently Json is a subset of YAML: cp foo.ipynb foo.ipyamlnb Yeah, Mission acomplished ! Second try¶ Install PyYaml, and see what we can do. In [42]: import json import yaml In [43]: from IPython.nbformat import current as nbf In [44]: ls Y*.ipynb YAML Notebook.ipynb In [45]: with open('YAML Notebook.ipynb') as f: nbook = nbf.read( f, 'json') In [46]: nbook.worksheets[0].cells[9] Out[46]: {u'cell_type': u'code', u'collapsed': False, u'input': u'from IPython.nbformat import current as nbf', u'language': u'python', u'metadata': {}, u'outputs': []} I'll skipp the fiddling around with the yaml converter. In short, you have to specify explicitely the part you want to dump in the literal form, otherwise they are exported as list of strings, which is a little painfull to edit afterward. I'm using the safe_dump and safe_load methods (or pass safeLoader and Dumper). Those should be default or otherwise you could unserialise arbitrary object, and have code exucuted. We probably don't want to reproduct the recent file Rail's critical vulnerability that append not so long ago. In [47]: # we'll patch a safe Yaml Dumper sd = yaml.SafeDumper # Dummy class, just to mark the part we want with custom dumping class folded_unicode(unicode): pass class literal_unicode(unicode): pass I know classes should be wit upper case, but we just want to hide the fact that thoses a class to end user. At the same time I define a folded method to use it with markdown cell. when markdown contain really long lines, those will be wrapped in the yaml document. In [48]: def folded_unicode_representer(dumper, data): return dumper.represent_scalar(u'tag:yaml.org,2002:str', data, style='>') def literal_unicode_representer(dumper, data): return dumper.represent_scalar(u'tag:yaml.org,2002:str', data, style='|') sd.add_representer(folded_unicode, folded_unicode_representer) sd.add_representer(literal_unicode, literal_unicode_representer) with open('YAML Notebook.ipynb') as f: nbjson = json.load(f) now we patch the part of the ipynb file we know we want to be literal or folded In [49]: for tcell in nbjson['worksheets'][0]['cells']: if 'source' in tcell.keys(): tcell['source'] = folded_unicode("".join(tcell['source'])) if 'input' in tcell.keys(): tcell['input'] = literal_unicode("".join(tcell['input'])) In [50]: with open('Yaml.ipymlnb','w') as f: f.write(yaml.dump(nbjson, default_flow_style=False, Dumper=sd)) You can round trip it to json, and it's still a valid ipynb file that can be loaded. Haven't fiddled with it much more. There are just a few gotchas with empty lines as well as trailing whitespace at EOL that can respectively diseapear or make the dumper fall back to a string quoted methods to store values. You can skip down to the end of this notebook to look at how it looks like. It's probably much compact than the current json we emit, in some cases it might be more easy to read, but I don't think it is worth considering using in the format specification. ipynb files are ment to be humanely fixable, and I strongly prefere having a consistent format with simple rules than having to explain what are the meaning of the differents shenigan like : |2+ for literal string. Also support across languages are not consistent, and it would probably be too much of a security burden for all code that will support loading ipynb to take care of sanitazing Yaml. One area where I woudl use it would be to describe the ipynb format at a talk for example, and/or to have metadata editing more human readable/writable. In [51]: !cat Yaml.ipymlnb metadata: name: YAML Notebook nbformat: 3 nbformat_minor: 0 worksheets: - cells: - cell_type: heading level: 1 metadata: {} source: >- YAML IPython notebook - cell_type: markdown metadata: {} source: "Little experiment base on the fact that apparently YAML is made to be\ \ better readable by Humans than JSON.\nWe've also had some complaint that metadata\ \ are not keep in nbconvert when roundtripping through markdown, those two\n\ made me think that I could try to see what ipynb files stored as YAML would\ \ look like. " - cell_type: heading level: 4 metadata: {} source: >- First atempt - cell_type: markdown metadata: {} source: >- Apparently Json is a subset of YAML: - cell_type: markdown metadata: {} source: >2+ cp foo.ipynb foo.ipyamlnb - cell_type: markdown metadata: {} source: >- Yeah, Mission acomplished ! - cell_type: heading level: 4 metadata: {} source: >- Second try - cell_type: markdown metadata: {} source: "Install PyYaml, and see what we can do. " - cell_type: code collapsed: false input: |- import json import yaml language: python metadata: {} outputs: [] - cell_type: code collapsed: false input: |- from IPython.nbformat import current as nbf language: python metadata: {} outputs: [] - cell_type: code collapsed: false input: |- ls Y*.ipynb language: python metadata: {} outputs: [] - cell_type: code collapsed: false input: |- with open('YAML Notebook.ipynb') as f: nbook = nbf.read( f, 'json') language: python metadata: {} outputs: [] - cell_type: code collapsed: false input: |- nbook.worksheets[0].cells[9] language: python metadata: {} outputs: [] - cell_type: markdown metadata: {} source: >- I'll skipp the fiddling around with the yaml converter. In short, you have to specify explicitely the part you want to dump in the literal form, otherwise they are exported as list of strings, which is a little painfull to edit afterward. I'm using the `safe_dump` and `safe_load` methods (or pass safeLoader and Dumper). Those should be default or otherwise you could unserialise arbitrary object, and have code exucuted. We probably don't want to reproduct the recent file Rail's critical vulnerability that append not so long ago. - cell_type: code collapsed: false input: |- # we'll patch a safe Yaml Dumper sd = yaml.SafeDumper # Dummy class, just to mark the part we want with custom dumping class folded_unicode(unicode): pass class literal_unicode(unicode): pass language: python metadata: {} outputs: [] - cell_type: markdown metadata: {} source: >- I know classes should be wit upper case, but we just want to hide the fact that thoses a class to end user. At the same time I define a folded method if I want to use it later. - cell_type: code collapsed: false input: |- def folded_unicode_representer(dumper, data): return dumper.represent_scalar(u'tag:yaml.org,2002:str', data, style='>') def literal_unicode_representer(dumper, data): return dumper.represent_scalar(u'tag:yaml.org,2002:str', data, style='|') sd.add_representer(folded_unicode, folded_unicode_representer) sd.add_representer(literal_unicode, literal_unicode_representer) with open('YAML Notebook.ipynb') as f: nbjson = json.load(f) language: python metadata: {} outputs: [] - cell_type: markdown metadata: {} source: >- now we patch the part of the ipynb file we know we want to be literal or folded - cell_type: code collapsed: false input: |- for tcell in nbjson['worksheets'][0]['cells']: if 'source' in tcell.keys(): tcell['source'] = folded_unicode("".join(tcell['source'])) if 'input' in tcell.keys(): tcell['input'] = literal_unicode("".join(tcell['input'])) language: python metadata: {} outputs: [] - cell_type: code collapsed: false input: |- with open('Yaml.ipymlnb','w') as f: f.write(yaml.dump(nbjson, default_flow_style=False, Dumper=sd)) language: python metadata: {} outputs: [] - cell_type: markdown metadata: {} source: >- You can round trip it to json, and it's still a valid ipynb file that can be loaded. Haven't fiddled with it much more. There are just a few gotchas with empty lines as well as trailing whitespace at EOL that can respectively diseapear or make the dumper fall back to a string quoted methods to store values. One could also try to tiker with `folded_unicode` in markdown cell that tipically have long lines to play a little more nicely with VCS. - cell_type: markdown metadata: {} source: >- You can skip down to the end of this notebook to loko at how it looks like. It's probably much compact than the current json we emit, in **some** cases it might be more easy to read, but I don't think it is worth considering using in the format specification. ipynb files are ment to be humanely fixable, and I strongly prefere having a consistent format with simple rules than having to explain what are the meaning of the differents shenigan like `: |2+` for literal string. Also support across languages are not consistent, and it would probably be too much of a security burden for all code that will support loading ipynb to take care of sanitazing Yaml. One area where I woudl use it would be to describe the ipynb format at a talk for example, and/or to have metadata editing more human readable/writable. - cell_type: code collapsed: false input: |- !cat Yaml.ipymlnb language: python metadata: {} outputs: [] metadata: {}

IPython Notebook Duck-Punching¶ If it walks like a duck and talks like a duck, it’s a duck. So if this duck is not giving you the noise that you want, you’ve got to just punch that duck until it returns what you expect. Small blogpost to answer to one question that has been asked on stackoverflow and on the Issue tracker : When I open a saved IPython Notebook, I need to evaluate all the cells with imports, function definitions etc. to continue working on the session. It is convenient to click Cell > Run All to do this. But what If I do not want to re-evaluate all calculations? Do I need to pick the cells to evaluate by hand each time? [There is] the concept of "initialization cells". You can mark some cells in the notebook as initialization cell, and then perform "evaluate initialization cells" after opening the notebook. This feature should allow certain cells to be marked as Initialization Cells and be evaluated together with the appropriate command. I'll let you get there to read the official answer, but in short : Not in core IPython, this can be done as an extension. Some warning before we start. As long as you don't shut down the IPython webserver, or don't ask for explicit shutdown, a notebook kernel stay alive, meaning that you can leave the page and come back, you wil still have the same active namespace. So you might not want to run those init cell again. Second do not ever, in any case, whatever reason you have, automatically run initialisation cell on notebook load. It is a security risk : If you have such an extension and I send you a notebook with a Initialisation Cell that have rm -rf ~/ in it. You just lost your home folder when openning this notebook. So this will show you how to to that in less than 60 lines of javascript, you might want to take a look at previous post for some info. You know where to find the finished code, as usual it is not perfect, and I wait for PR to fix the edge cases. Let's start. Overview¶ This extension will be in two part even the all thing will fit in one file. So let's decompose what we need. The ability to mark a cell with a certain flag (Initialisation Cell) The ability run all those initialisation cell when needed For me this sound like the need for a custom CellToolbar checkbox, and a Toolbar button 'Run init Cell'. A checkbox because this is typically a Boolean statement, the cell is a Initialisation cell, or not. And the toolbar button is the easiest to reach, and not too complicated to add. The CellToolbar checkbox¶ We will store the boolean of wether of not the cell is an Initialisation Cell in the cell Metadata. To stay clean, we will use a prefixed key to avoid future collision. As this is a draft, I will prefix the name of the key with an underscore to warn future user that directly accessing this key is not supported and that I reserved myself the right to change anything without warning. I will store [true|false|undefined] in cell.metadata._draft.init_cell. I will not forget to check that cell.metadata._draft exist before playing with it. The IPython notebook provide a convenient API to generate checkbox in Cell Toolbar. to use this we need to define a getter and a setter for our metadata. The setter take the cell we act on, and the new value: function(cell, value){ // we check that the _draft namespace exist and create it if needed if (cell.metadata._draft == undefined){cell.metadata._draft = {}} // set the value cell.metadata._draft.init_cell = value } The getter is not much more complicated : function(cell){ var ns = cell.metadata._draft; // if the _draft namespace does not exist return undefined // (will be interpreted as false by checkbox) otherwise // return the value return (ns == undefined)? undefined: ns.init_cell } The api to generate the checkbox has the following signature : CellToolbar.utils.checkbox_ui_generator(label, setter, getter) I can then create my function easily: var CellToolbar= IPython.CellToolbar; var init_cell = CellToolbar.utils.checkbox_ui_generator('Initialisation Cell', // setter function(cell, value){ // we check that the _draft namespace exist and create it if needed if (cell.metadata._draft == undefined){cell.metadata._draft = {}} // set the value cell.metadata._draft.init_cell = value }, //getter function(cell){ var ns = cell.metadata._draft; // if the _draft namespace does not exist return undefined // (will be interpreted as false by checkbox) otherwise // return the value return (ns == undefined)? undefined: ns.init_cell } ); The label will be use in the UI to put a name in front of the checkbox to know its use. So I used a descriptive name. Now we need to register our function, for that we will use CellToolbar.register_callback(name, function);. name should be a string we will use to refer to the function later, in order to use it in multiple place if we wish. Here simply CellToolbar.register_callback('init_cell.chkb', init_cell); And finaly, we use a private method (for now) to generate a CellToolbar preset that can be chosen by the User in the CellToolbar dropdown with : CellToolbar.register_preset(label, ['callback_name','callback_name',....]). This allow to simply mix and match ui elements from different preset for customisation. Here we only have one checkbox so we do: CellToolbar.register_preset('Initialisation Cell', ['init_cell.chkb']); With all the extension I have, I could create a custom CellToolbar simply by adding: CellToolbar.register_preset('My toolbar', ['init_cell.chkb','default.rawedit','slideshow.select']); And you can see below how it looks like In [19]: from IPython.display import Image Image(filename='/Users/bussonniermatthias/Desktop/Ctoolbar.png') Out[19]: Now you just need to select the Initiallisation Cell CellToolbar and check the checkbox you wish. The Toolbar button¶ Now we need a way to run all cells marked as Initialisation Cells. Let's first make a function that loop on all cell and run is they are marked: var run_init = function(){ var cells = IPython.notebook.get_cells(); for(var i in cells){ var cell = cells[i]; var namespace = cell.metadata._draft|| {}; var isInit = namespace.init_cell; // you also need to check that cell is instance of code cell, // but lets keep it short if( isInit === true){ cell.execute(); } } }; Now we use the API to create a function that register a callback on a button click on the main toolbar, we use a descriptive label that shows on hower, on Icon from jQuery UI themeroller, and assign the callback to previous defined function: var add_run_init_button = function(){ IPython.toolbar.add_buttons_group([ { 'label' : 'run init_cell', 'icon' : 'ui-icon-calculator', 'callback': run_init } ]); }; Now we just need to run this function late enough to have effect. We will just listen for the loaded even to trigger this : $([IPython.events]).on('notebook_loaded.Notebook',add_run_init_button); That's it. You just have to put all this stuff in one big file, name it correctly and add the $.getScript in your custom.js. Reload your page/Restart your server (depending on the config you choose). And you should have a new toolbar preset and a button to run your initialisation cell. I've got less than 60 lines of javascript comment and blank lines counted. As this is a separated file, you can easily fork it and add modifications/options. I'm waiting for PRs. I hope this show you that writing extension for the IPython notebook is not that hard, and are easy to share. In [ ]:

IPython Notebook Duck-Punching (II)¶ If it walks like a duck and talks like a duck, it’s a duck. So if this duck is not giving you the noise that you want, you’ve got to just punch that duck until it returns what you expect. Some thoughts on the IPython notebook. For once, this will discuss more what are the possible things than can be done with IPython, with only few really usable code. I hope it will help the reader to better understand in what direction the Core Team whant to drive IPython. This is of course my personnal interpretation of the different points. We discussed since I came on board. But I hope it will help the comunity to feel.concern and participate. Terminology¶ I think one of the first things that should be adressed is the terminology of things. As a non native english speaker I am pretty bad at choosing names for things. And I probably have an horible.english. And I think there is a big problem the Naming in the IPython notebook. The problem is the same as what happend with the terme "gene", at some point in history it was used to explain a specific effect that some phenotype where expressed depending on the genes you had, the problem is that with our current technologie an knowledge on genetics, the boundaries of what a gene is becommes fuzzy. Same apply to the IPython notebook. Let's see. You probably can start the IPyrhon notebook from your command line. You can also ask a friend to send his or her IPython notebook by email. You probably know that shift enter execute a cell in an IPython notebook. But you probably don't send a webserver by email. I haven't seen anyone converting GMail to a power point. I think you see the point. "IPython notebook" represent at the same time: The webserver application that yoi start from the command line, the web frontend that you are used to use, the file format that defines the structures of ipynb files, the actual files you exchanges, and the representations of thoses files into the web applocation. Add on top the fact that kernel can speek to multiple frontend and can talk to non-python kernel, and people get lost. Now you wonder why you should care as you are only using the all bundle? Because you will probably what to share and or reuse what you write, and once you understand the differents things.you will be able to do even more amazing things. The part you don't (really) care about. The webserver. It can mostly be seen as a middleware. The configuration options should be left to the sysadmin. It mostly make the ZMQ/websocket bridge, and is also an endpoint for the web application to access remote filesystem (yes remote, even if localhost). The part you can do without. The kernel. You probably had one or twice a kernel that died In [ ]:

IPython Notebook Duck-Punching (II)¶ If it walks like a duck and talks like a duck, it’s a duck. So if this duck is not giving you the noise that you want, you’ve got to just punch that duck until it returns what you expect. Response to comments¶ I might have been unclear in previous post introduction about Better typography for IPython notebooks, I didn't ment to say that good typography and theme was a feature request and that we wouldn't do it. It is planned, it just need refactoring, and especially, it shoudln't be as difficult as it is now do developp a new theme. If developping a theme is easy, then we can do in parallel of Main IPython developpement, and we just have to make this theme the default one. But right now, changing css is a real pain with all the browser os and configuration around. I also was a little too optimistic on the necessary version to have custom.css working. It aapparently appeard after the 0.13 branch but was not backported to 0.13.1, so you will need developpemetn version to have previous post (and this one) fully working. TL;DR:¶ Last wee we saw custom.css is the way to inject css, today we'll use custom.js to asynchronouly fetch css and replace the default one. The rest is api detail and rely wrapping arount the following jquery call : $('link:nth(7)').attr('href','/static/css/new-style.min.css') Intro¶ This the second nbviewer post on the IPython notebook. Mainly in responses to comment on Better typography for IPython notebooks, we saw last time how to locally add custom css to a notebook wihtout too much pain. Now we are going to dg deeper into IPython code and see what we can do. As for the first post, you will stil be able to do everything without changing the source, but still need to get the dev version. For those of you on linux; you can get a daily build with Julian Taylor PPA that should work great. Be carefull though, it has a few custom patches that moved the static resources into /usr/share/ This notebook will contain javascript snippet, but unlike last post, I will not put them in %%javascript magic for a few reasons, the first beeing that it will not work on nbviewer, the second is that you should stop relying on publishing javascipt using the _repr_javascript_ of IPyton as it will be deprecated. This will be the subject of another post, but don't complain we have bigger plan to replace that. Start Punching¶ Warning¶ I'm dooing this post on a dev version, so some stuff might differs, especially in some places you might need to use $('link:nth(5)') instead of $('link:nth(7)') and vice versa. Let's carry on... If you made your home work, you probably know that custom.js is in profile_xxx/static/js/custom.js and that any file in profile_xxx/static/[dirs]/[name] is availlable with the /static/[dirs]/[name] url once in notebook. Here is the doc for custom js : Placeholder for custom user javascript mainly to be overridden in profile/static/js/custom.js This will always be an empty file in IPython User could add any javascript in the profile/static/js/custom.js file (and should create it if it does not exist). It will be executed by the ipython notebook at load time. Same thing with profile/static/css/custom.css to inject custom css into the notebook. Example : Create a custom button in toolbar that execute %qtconsole in kernel and hence open a qtconsole attached to the same kernel as the current notebook $([IPython.events]).on('notebook_loaded.Notebook', function(){ IPython.toolbar.add_buttons_group([ { 'label' : 'run qtconsole', 'icon' : 'ui-icon-calculator', 'callback': function(){IPython.notebook.kernel.execute('%qtconsole')} } // add more button here if needed. ]); }); Example : ... You can view the doc using yuidoc than run with node and can be installed via npm, running yuidoc --server in the js directory will provide you with the most up to date documentation, and you are welcomed to bring correction and improvement to it, as well as helpin us having a daily build. As custom.css help us to inject css into the web-notebook, custom.js allo us to inject custom javascript. And you might guess maintaining a lot of javascript into one file is painfull, so we will just use that as en entry point. Little side note : For those of you that don't do javascript, the dollar sign $ is a valid variable name and is usually bind to jQuery ( a library ) that do many things. So $(foo).bar() is calling jQuery with the parameter foo (most of the time a selector) and then calling the method bar on $(foo). Wherease $.foo is calling the method foo of jQuery itself. Don't worry about the exact meaning or difference, just remember that $ is an object that does things and not something part of JS language, and that it is often refered in text as jQuery. So we can use jQuery getScript to just load another file where we will work. In [5]: %%bash touch ~/.ipython/profile_default/static/js/css_selector.js In [25]: %%file /Users/bussonniermatthias/.ipython/profile_default/static/js/css_selector.js console.log("I'm loaded") Overwriting /Users/bussonniermatthias/.ipython/profile_default/static/js/css_selector.js Be carefull the %%file magic does not warn before replacing a file Now we load dynamicaly this file, adding the following to our `custom.js`` $.getScript('/static/js/css_selector.js') I cheat a little and don't do it from the notebook as my config file have more stuff in it, but here is the corresponding line. In [20]: !cat ~/.ipython/profile_default/static/js/custom.js | head -n 6 | tail -n 1 $.getScript('/static/js/css_selector.js') Now you can run your notebook and you should see a message in the javascript console that say "I am loaded". So now, the big seecret is to find the current css file and replace the link to our custom link. This is the first feature that is not supported (yet ?) by the notebook and you will need to adapt to this durring the developpement. With time things will become easier to do hopefully So, select the 5th (or 7th) in which contain the current style. You can do that on the js console. $('link:nth(5)') Check that it give you : [] You can now set the new css with the following command: $('link:nth(5)').attr('href','/static/css/new-style.min.css') But wait, if you replace the default style won't this screw everything up by removing the default style ? Yes, That why know we'll see how to compile your own full style :-) Isn't it the goal of thoses posts ? Recompilying a style¶ You can if you wish create a style from scratch, this does not require any more tool. It will just probably be painfull, so I suggest you start by modifying the IPython ones to create a new theme and compile it. To do so you'll need some extra dependencies python fabric, lessc, bower and a few other that bower will install. Get the IPython dev source tree, and go into IPython/frontent/html/notebook/ and issue a $ bower install It will look for all the dependencies in component.json and install them. This include bootstrap for the css and a few more things, that will be installed in the componentdirectory. The IPython repo will move more and more toward requiring things as "component" later, especially for dev version. You can now use less or lessc compiler to turn all files in /static/less/*.less into one big css theme file. this can be done with : $ lessc -x less/style.less output_file or : $ fab css to regenerate IPython one. Now this is more or less up to you, I advise you to "fork" static/less/style.less and static/less/varaibles.less then recompile to get your new theme. Those file will probably change a lot during the next few month. And the is typically the place where we need your help to refactor. You can also go to the end of this post where I link to a few ugly themes. Wrinting the selector¶ Using a dropdown list to select the theme we want seem to be the correct UI choice, but writing api to insert select/options element in toolbar woudl be more complicated than dooing it by hand. For simpler action that require a button, astonishly the required code is more difficult, hence we provide some help methods to do so. I'll let you hit the docs for that. As writing the selector is not the interesting exercise, I'll just give you the code to put in your css_selector.js: In [ ]: var add_css_list = function (element,names) { console.log(element) var label = $('').text('Css:'); var select = $('') .addClass('ui-widget-content') .append($('').attr('value', 'style.min').text('default')); element.append(label).append(select); select.change(function() { var val = $(this).val() $('link:nth(5)').attr('href','/static/css/'+val+'.css') /*********************/ // missing stuff Here /*********************/ }); for (var i=0; i').attr('value', name).text(name)); } }; add_css_list(IPython.toolbar.element,['dark','duck']) This does the basic job, it changes the notebook css. I tested 2 themes here, "dark" and "duck", in which I only change the following : @corner_radius: 0px/0px; @notebook_background : yellow/balck; @borderwidth : 3px; @fontBaseColor : black/white; and compiled. In [5]: from IPython.display import Image Image(filename='/Users/bussonniermatthias/Desktop/d-and-d.png') Out[5]: look closely, some buttons are unusual for a notebook... So as you can see, not all the css does support variable right now, and if you know a little how the IPython notebook works, we use CodeMirror as an edittor and you might want to set the theme of all codemirror edittor at the same time. Hence the /*********************/ // missing stuff Here /*********************/ in the above snippet. One should both set the default CodeMirror theme on all cell, but also change the actual with: editor.setOption("theme", theme); as well as load the corresponding css. This probably need a loop on IPyhton.notebook.get_cells() Any way, I hope this show you that you can do a lot with the IPython notebook without diving deep into the source code. For the Lazy reader, the needed code and css files are availlabe here, I'll probably move it in a more appropriate place later. You will still need to find the right place where to copy it, and add aline in your config.js (how hard!). Feel free to send me PR to improve this code/ share new css. Next time¶ Next time we'll probably go back a little more on (I)Python side or explore some thought on what can be done on the Metadata in the notebook. I might also speek to you a little about nbconvert and the philosophy behind the notebook format. If you have any comment, feel free to comment on the gihub repo of this blog post, and/or to send me PR to fix mistakes.

IPython Notebook Duck-Punching (I)¶ If it walks like a duck and talks like a duck, it’s a duck. So if this duck is not giving you the noise that you want, you’ve got to just punch that duck until it returns what you expect. This will try to be a series of blog post (or nbviewer post) on the IPython notebook. Mainly in responses to comment on Better typography for IPython notebooks and some of the comment at the end of the page, especially this one, that will have a full answer next post. First some presentation, I am Matthias, you can usually find me on github aka @carreau, or on twitter @mbussonn. I'm a Phd student in Biophysic, more physic than bio. And I contributed mostly to the IPython notebook. This would also probably be the starting point for the IPython Advance tutorial I'll be giving next August in Euro SciPy. Addendum¶ I was a little optimistic in presupposing that custom.css was availlable in 0.13.1, so most of what is discussed here need a more recent developpement version to work. A more general issue¶ The question about changing the style of the notebook come often and it is part in my opinion of the question of type : Why dont you want to integrate X? Why is it not implemented? I would like to take some time to respond to it, in both a general way and try to introduce to to some concept of the notebook few people know about. Brian Granger already did some good post about it here and here. As I am also responsible for the refactoring of nbconvert and are the principal maintainer of nbviewer I'll try to share my thought on what the future will be, and what is currently in progress. So, back to our question, why do we not implement some feature. Well, a non negligeable part of the time, because they already exist, you are just not aware of that. Plese read the doc (or say you did) and ask how to do it because you didn't well understood. Once you acchieve to do it, a good way to help us and other is to improve the doc, Pull Request are welcomed :-) Second possibility: You don't need code in the core to do it. We provide more or less easy way to hook onto IPython. And as there a are many reason for which some things are better outside of IPython than in the core, we will encourage you to do ship it separately. Third: we won't special case only for you. This does not mean we don't like what you did, more often it mean that the internal of IPython shoudl allow what you did to be an extension, they just don't allow it yet. Bare with us for some time or help us to acchieve our goal and it will be better for everyone soon ! What about css theming ?¶ Short answer : I haven't seen any requests about theming that cannot be done without patching IPython code in itself. Some part could be made much easier, and it is in progress. What you ask (Custom theme, share them...) is already here, you just don't know how to do it. Carl said : Warning: I don’t know what I’m doing. Don’t make any of these changes, or any others, without backing up the files first. But me, I do know what I'm dooing when speeking of IPython, but not when speaking of design, so let's start to tinker with IPython to have custom css, and easy to share. First thing to remember if you ever think of modifying a file in IPython/notebook/html/static/* you are dooing it wrong. Let's dive a little into how to customise the IPython notebook. Adding custom css:¶ How to add custom css to notebook, starting by the wrong way. Things you must never do. Modifyin IPython source files.¶ Never do that, except if you want to open a pull request to fix a bug. Actually you should never modify a file which is under IPython/frontend/html/notebook/static/* because you don't need to do it. We'll se later why and what to do. This mean, that if you are not admin on your machine, or you just don't want to modify the system file, you can still read the rest and try by yourself. CSS In a markdown cell¶ Probably the worse way to do it. You can create a markdown cell with style tag in it, and write some css that will apply to the notebook. It will most likely break on future version, you have to add it every time. You will bother other when sharing your notebook, and it will probably break the conversion process into pdf/rst/markdown when you use nbconvert Html markup will not be the same in nbviewer, so your post might be ugly, and if there are any update of something at some point, you will have to update all your .ipynb files. Even if this is great to test some css as a quick an dirty way (like I did for this notebook) I strongly advise not to do it. It is not yet clear with how things are right now, but a notebook is a document that contain data, and the frontend is responsible for the formatting. Right now the notebook server has few frontends: - the browser one - and the [emacs client](https://github.com/tkf/emacs-ipython-notebook). But this is likely to change, so please no style in Markdown cell. The answer : custom.css¶ So here we are, the right way to add custoom css to a notebook when you look at it throught the browser interface, use the custom.css file. This file is not created by default, and can exist on a per-profile basis, if you don't know what ipython profiles are, then you are probably using the default profile. In short, profile are a way to have different configuration for ipython which you can choose through a command line flag (ipython notebook --profile=). Let's locate our profile folder: In [2]: %%bash ipython locate /Users/matthiasbussonnier/.ipython This tells me that IPyton is expecting profiles in the above directory, and more specially on profile named foo will have the corresponding files in /Users/matthiasbussonnier/.ipython/profile_foo/ let's create a profile for the sake of this blog post. In [3]: %%bash ipython profile create customcss [ProfileCreate] Generating default config file: u'/Users/matthiasbussonnier/.ipython/profile_customcss/ipython_config.py' [ProfileCreate] Generating default config file: u'/Users/matthiasbussonnier/.ipython/profile_customcss/ipython_qtconsole_config.py' [ProfileCreate] Generating default config file: u'/Users/matthiasbussonnier/.ipython/profile_customcss/ipython_notebook_config.py' This created the needed folder structure for IPython to work, but we won't be interested in those file for now. If you do not want to create a custom profile, you could also modify the files in profile_default which is the profile IPython uses when nothing is specified. I will now create the file I am interested in : static/custom/custom.css In [5]: %%bash mkdir ~/.ipython/profile_customcss/static/ mkdir ~/.ipython/profile_customcss/static/custom/ touch ~/.ipython/profile_customcss/static/custom/custom.css Use the file magic to write something in it. In [34]: %%file /Users/matthiasbussonnier/.ipython/profile_customcss/static/custom/custom.css /**write your css in here**/ /* like */ Overwriting /Users/matthiasbussonnier/.ipython/profile_customcss/static/custom/custom.css In [35]: cat ~/.ipython/profile_customcss/static/custom/custom.css /**write your css in here**/ /* like */ Now every time you start ipython with : $ ipython notebook --profile customcss [NotebookApp] Using existing profile dir: u'~/.ipython/profile_customcss' [NotebookApp] Serving notebooks from local directory: /Users/matthiasbussonnier/ [NotebookApp] The IPython Notebook is running at: http://:8889/ [NotebookApp] Use Control-C to stop this server and shut down all kernels. You will get the right css, let's try : In [36]: %%bash curl --noproxy localhost http://localhost:8889/static/custom/custom.css /**write your css in here**/ /* like */ % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 240 100 240 0 0 102k 0 --:--:-- --:--:-- --:--:-- 234k Yeah ! We now get you custom css that will be loaded in notebook, without dangerous file modifications, and without using root rights ! Things to come.¶ On IPython notebook-server side¶ I must warn you, do not rely too much on current css and classes to make your custom theme. We are both refactoring and introducing new tools to make our (and your) life easier. We are progressively moving our css to bootrap, and we currently have part of it that is generated through a compilation of less file. This allow us to introduce css variables, so that you can, for example, set a global HUE for the theme an a radius for the corner, recompile, and you get your new theme ready. Just use it as a custom css in your profile dir and you are good to go. Here are one example of what you can do. And as a bonus, (I'll let you search) we added a notebook flag to compile css on the fly in the browser, so you can develop your theme with a less folder wirhout triggering compilation yourself.  I told you I was not good in design. So now our notebook look more like an ugly duckling, but we now how to pet it so that it behave more like we want, and you can share it! On nbconvert/viewer side¶ The notebook format suppor metadata, so I don't see any reason not to set a prefered theme for a notebook whe viewing with a specific application/frontend. This might include nbviewer, it we consider that css are safe enough and got the time to add the concept of user to nbviewer I don't see any reason not to support external css. @damiamavilla already have build a slideshow version of nbviewer (that we will probably release in the next 6 month) that support multiple theme for the same notebook. And if you made some theme, feel free to share, I even think that a user-governed repository with multiple css woudl be great. Conclusion¶ Custom css is doable, and will improve, and the more you help us, the faster it will arrives ! Also this show you that this does not need to be part of IPython core to exist, and having it separately will allow faster release cycle or even rolling release of themes. If you have any comment corrections, I think you'll probably find the gist/github repo that correspond to this notebook. Next post trailer.¶ Custom.css is not the only file that you can use to inject css in the notebook. Actually any file that you ask to the webserver that start with the /static/ prefix will be first searched in your profile dir. You can also add path with NotebookApp.extra_static_paths= configuration option. So as you'll have guessed, custom.css exist in the directory where the static resources are installed, it only contains comments. Next time, we will use custom.js (I think you can find it by yourself) as an entry point into the notebook to load more javascript dynamically and look at where we can hook to create a css selector. I'll dive into the recent api we added to javascript, and what are the great things you can do with it. I'll let you prepare a few themes to play with, feel free to share them on ipython-contrib. Not many javascript knowledge will required, you just need to find the curly bracket on your keyboard.