Caravel - sweet time based visualisation and dashboard reporting - written in python! Source

April 6, 2016 Markdown source

1---
2title: 'Caravel - sweet time based visualisation and dashboard reporting - written in python!'
3date: '2016-04-06'
4published_at: '2016-04-06T19:34:00.001+10:00'
5tags: ['airbnb', 'databases', 'open source', 'python', 'reporting', 'software']
6author: 'Gavin Jackson'
7excerpt: 'The fine folks at airbnb recently opensourced an amazing software product called caravel. It is a web-based data visualisation and dashboarding application. It looks amazing and I hope to share some o...'
8updated_at: '2016-04-06T19:34:56.122+10:00'
9legacy_url: 'http://www.gavinj.net/2016/04/caravel-sweet-time-based-visualisation.html'
10---
11
12[![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHL35EvwSpZRjLl4ZUxnyXBRbbTN0B4cfXxFwY7oHBHiM8ETBVVI5bXXKRn_J3K4641fBff59LsnduvGVKTGS_CAJOWSIRe-he85JAjRU00EFy40YyLBCGrUtPaxuBed5Cq7kCSx9kGuA/s320/caravel_screenshot_1.png)](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHL35EvwSpZRjLl4ZUxnyXBRbbTN0B4cfXxFwY7oHBHiM8ETBVVI5bXXKRn_J3K4641fBff59LsnduvGVKTGS_CAJOWSIRe-he85JAjRU00EFy40YyLBCGrUtPaxuBed5Cq7kCSx9kGuA/s1600/caravel_screenshot_1.png)
13
14The fine folks at airbnb recently opensourced an amazing software product called caravel. It is a web-based data visualisation and dashboarding application. It looks amazing and I hope to share some of my experiences getting it up and working.
15
16https://github.com/airbnb/caravel
17
18Things I love about this tool:
19
201. It's 100% open source - no "paid" or "locked" features - what you see is what you get.
212. It's a big, beautiful python application, I think it is amazing to see a product of such scale that uses libraries that I use on a day to day basis - I'm looking forward to reading the source code of this app!
223. Although the documentation is light on (presently), it does work as described - it's not just a marketing gimmick - and they have a very cool looking road map for future improvements (including python 3 support!).
234. It fills a niche that other open source reporting tools (such as Jasper and Pentaho) don't do well - time-based tactical views on data.
245. The sqlalchemy support means that most databases are supported out of the box.
256. The druid integration provides future expansion for large/aggregated datasets (you can see why airbnb would be interested in this).
26
27Things that could be improved:
28
291. It does not support table joins out of the box - which means that I'm going to have to set up some views in my database to produce meaningful reports.
302. All of the graphs seem to require time based data - not everything i want to report on has a temporal element.
313. The spatial support is pretty rudimentary, it's really based on ISO country code - I would imagine that it is beyond the scope of the project to hook in to postgis and supply an openlayers widget (but what a possibility!).
32
33What an amazing open source tool - bravo to the developers and to airbnb for releasing this for us to use and expand!
34
35[![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbVMo7lJuyhnuaXiKRev2uNO8A1_B0bKmyh9ONUxfo6S4EhW9-SUBxrs4nTQ3yFDE_67-g2k2UwV6NB4hTtgUfrPd9xNIkmVQgMwdFTzp_LAhHOnV2Rhh9aX8XmYNOjYNd_vwRiwaB2TM/s400/caravel_screenshot_2.png)](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbVMo7lJuyhnuaXiKRev2uNO8A1_B0bKmyh9ONUxfo6S4EhW9-SUBxrs4nTQ3yFDE_67-g2k2UwV6NB4hTtgUfrPd9xNIkmVQgMwdFTzp_LAhHOnV2Rhh9aX8XmYNOjYNd_vwRiwaB2TM/s1600/caravel_screenshot_2.png)
36
37##  Installing caravel on Mac OSX 10.11.4 *El Capitan*
38
39Below are my OSX installation notes (hopefully some folks out there may find them useful).
40
41Firstly it is currently python 2.7 only which means you have to re-learn how to use virtualenv. On Linux this is pretty straight forward, on OSX it required a bit of mucking around.
42
43**Step 1 - Install xcode (via app store)**
44
45then run xcode-select --install
46
47**Step 2 - install home-brew**
48
49Old versions can be problematic (so you can uninstall and reinstall if necessary)
50
51`ruby -`e "$(curl -fsSL [https://raw.githubusercontent.com/Homebrew/install/master/uninstall](https://raw.githubusercontent.com/Homebrew/install/master/uninstall))"
52
53`ruby`-e "$(curl -fsSL [https://raw.githubusercontent.com/Homebrew/install/master/install](https://raw.githubusercontent.com/Homebrew/install/master/install))"
54
55**Step 3 - install python 2.7 (caravel does not support python 3 … yet)**
56
57I then had to fix the python path (as it kept pointing to /usr/bin)
58
59brew link —overwrite python
60
61**Step 4 - install virtualenv**
62
63pip install virtualenv
64
65**Step 5 - create virtualenv**
66
67(note: this is the old python 2.7 way of doing things, python 3 introduced pyvenv)
68
69virtualenv venv_caravel
70
71**Step 6 - activate virtualenv**
72
73../venv_caravel/bin/activate
74
75**Step 7 - follow the caravel installation instructions**
76
77** **
78
79I had to run the following to successfully install the cryptography library:
80
81env LDFLAGS="-L$(brew --prefix openssl)/lib" CFLAGS="-I$(brew --prefix openssl)/include" pip install cryptography
82
83# Install caravel
84pip install caravel
85
86# Create an admin user
87fabmanager create-admin --app caravel
88
89# Initialize the database
90caravel db upgrade
91
92# Create default roles and permissions
93caravel init
94
95# Load some data to play with
96caravel load_examples
97
98# Start the development web server
99caravel runserver -d
100
101
102

---
title: 'Caravel - sweet time based visualisation and dashboard reporting - written in python!'
date: '2016-04-06'
published_at: '2016-04-06T19:34:00.001+10:00'
tags: ['airbnb', 'databases', 'open source', 'python', 'reporting', 'software']
author: 'Gavin Jackson'
excerpt: 'The fine folks at airbnb recently opensourced an amazing software product called caravel. It is a web-based data visualisation and dashboarding application. It looks amazing and I hope to share some o...'
updated_at: '2016-04-06T19:34:56.122+10:00'
legacy_url: 'http://www.gavinj.net/2016/04/caravel-sweet-time-based-visualisation.html'
---

[![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHL35EvwSpZRjLl4ZUxnyXBRbbTN0B4cfXxFwY7oHBHiM8ETBVVI5bXXKRn_J3K4641fBff59LsnduvGVKTGS_CAJOWSIRe-he85JAjRU00EFy40YyLBCGrUtPaxuBed5Cq7kCSx9kGuA/s320/caravel_screenshot_1.png)](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHL35EvwSpZRjLl4ZUxnyXBRbbTN0B4cfXxFwY7oHBHiM8ETBVVI5bXXKRn_J3K4641fBff59LsnduvGVKTGS_CAJOWSIRe-he85JAjRU00EFy40YyLBCGrUtPaxuBed5Cq7kCSx9kGuA/s1600/caravel_screenshot_1.png)

The fine folks at airbnb recently opensourced an amazing software product called caravel. It is a web-based data visualisation and dashboarding application. It looks amazing and I hope to share some of my experiences getting it up and working.

https://github.com/airbnb/caravel

Things I love about this tool:

1. It's 100% open source - no "paid" or "locked" features - what you see is what you get.
2. It's a big, beautiful python application, I think it is amazing to see a product of such scale that uses libraries that I use on a day to day basis - I'm looking forward to reading the source code of this app!
3. Although the documentation is light on (presently), it does work as described - it's not just a marketing gimmick - and they have a very cool looking road map for future improvements (including python 3 support!).
4. It fills a niche that other open source reporting tools (such as Jasper and Pentaho) don't do well - time-based tactical views on data.
5. The sqlalchemy support means that most databases are supported out of the box.
6. The druid integration provides future expansion for large/aggregated datasets (you can see why airbnb would be interested in this).

Things that could be improved:

1. It does not support table joins out of the box - which means that I'm going to have to set up some views in my database to produce meaningful reports.
2. All of the graphs seem to require time based data - not everything i want to report on has a temporal element.
3. The spatial support is pretty rudimentary, it's really based on ISO country code - I would imagine that it is beyond the scope of the project to hook in to postgis and supply an openlayers widget (but what a possibility!).

What an amazing open source tool - bravo to the developers and to airbnb for releasing this for us to use and expand!

[![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbVMo7lJuyhnuaXiKRev2uNO8A1_B0bKmyh9ONUxfo6S4EhW9-SUBxrs4nTQ3yFDE_67-g2k2UwV6NB4hTtgUfrPd9xNIkmVQgMwdFTzp_LAhHOnV2Rhh9aX8XmYNOjYNd_vwRiwaB2TM/s400/caravel_screenshot_2.png)](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbVMo7lJuyhnuaXiKRev2uNO8A1_B0bKmyh9ONUxfo6S4EhW9-SUBxrs4nTQ3yFDE_67-g2k2UwV6NB4hTtgUfrPd9xNIkmVQgMwdFTzp_LAhHOnV2Rhh9aX8XmYNOjYNd_vwRiwaB2TM/s1600/caravel_screenshot_2.png)

##  Installing caravel on Mac OSX 10.11.4 *El Capitan*

Below are my OSX installation notes (hopefully some folks out there may find them useful).

Firstly it is currently python 2.7 only which means you have to re-learn how to use virtualenv. On Linux this is pretty straight forward, on OSX it required a bit of mucking around.

**Step 1 - Install xcode (via app store)**

then run xcode-select --install

**Step 2 - install home-brew**

Old versions can be problematic (so you can uninstall and reinstall if necessary)

`ruby -`e "$(curl -fsSL [https://raw.githubusercontent.com/Homebrew/install/master/uninstall](https://raw.githubusercontent.com/Homebrew/install/master/uninstall))"

`ruby`-e "$(curl -fsSL [https://raw.githubusercontent.com/Homebrew/install/master/install](https://raw.githubusercontent.com/Homebrew/install/master/install))"

**Step 3 - install python 2.7 (caravel does not support python 3 … yet)**

I then had to fix the python path (as it kept pointing to /usr/bin)

brew link —overwrite python

**Step 4 - install virtualenv**

pip install virtualenv

**Step 5 - create virtualenv**

(note: this is the old python 2.7 way of doing things, python 3 introduced pyvenv)

virtualenv venv_caravel

**Step 6 - activate virtualenv**

../venv_caravel/bin/activate

**Step 7 - follow the caravel installation instructions**

** **

I had to run the following to successfully install the cryptography library:

env LDFLAGS="-L$(brew --prefix openssl)/lib" CFLAGS="-I$(brew --prefix openssl)/include" pip install cryptography

# Install caravel
pip install caravel

# Create an admin user
fabmanager create-admin --app caravel

# Initialize the database
caravel db upgrade

# Create default roles and permissions
caravel init

# Load some data to play with
caravel load_examples

# Start the development web server
caravel runserver -d