Map Fields - A Rails plugin to ease the importing of CSV files
I’ve just released a plugin, map-fieilds, that eases the importing of CSV files.
When importing CSV files for a project, I wanted to add flexibility for the users so that they could import their CSV files in a looser format and then map their format to the format I needed.
map-fieilds will intercept calls to a method and show an intermediate screen where the user can map their columns to the expected columns.
How to install
In your environment.rb file:
If you prefer, it can be installed as a plugin:
Using it in your controller
lists_controller.rb:
map_fields :create,
['Title', 'First name', 'Last name'],
:file_field => :file,
:params => [:list]
def index
@lists = List.find(:all)
end
def new
@list = List.new
end
def create
@list = List.new(params[:list])
if fields_mapped?
mapped_fields.each do |row|
@list.contact.create(:title => row[0],
:first_name => row[1],
:last_name => row[2])
end
flash[:notice] = 'Contact list created'
redirect_to :action => :index
else
render
end
rescue MapFields::InconsistentStateError
flash[:error] = 'Please try again'
redirect_to :action => :new
rescue MapFields::MissingFileContentsError
flash[:error] = 'Please upload a file'
redirect_to :action => :new
end
end
Explanation
Setup map-fields
Setup map-fields at the top of the controller.
map_fields accepts three parameters
- The first is the method to intercept, in this case :create
- The second is an array of expected CSV fields. The order of the fields in this array is the order they will be available in the row object when finally reading the CSV file.
- Lastly, a hash of options.
:file_field is the field that contains the import file. This is :file by default
:params is an array of parameters you want preserved. If you have a form based around a model, you just need to put the model name here and all the sub-fields will be preserved.
So, if you have a form with list[:name] etc, use :params => [:list]
Create your new view with a file field
You can now setup your new view as normal with an included file field
Handle the mapping in your create action
The create action now has to perform two functions, the mapping and then the final creating.
You can call the fields_mapped? method to see if the mapping has been performed and if not, render the mapping view.
There is a mapping partial which you can use so your view is as easy as:
<%= render :partial => 'map_fields/map_fields' %>
and it produces the following:
When the fields have been mapped, you can iterate through them with:
# row.number returns the number of the row in the original CSV file
# row[0] is the first mapped field, in this case Title
# row[1] is the second mapped field, in this case First name
# row[2] is the third mapped field etc...
end
Two errors can be raised:
- MapFields::InconsistentStateError is raised when map-fields is unable to determine whether a file is being uploaded or mapped. It can be experienced through a combination of using the back button and refreshes but is seldom experienced.
- MapFields::MissingFileContentsError is raised when no file has been uploaded
Please feel free to ask any questions in the comments and raise issues on the GitHub page.
[...] Map Fields - A Rails plugin to ease the importing of CSV files @ Ramblings on Railsramblingsonrails.com [...]
I can’t see any benefit over the usual and easy method of reading csv files. I think you’ve tried to solve a problem that doesn’t exist.
Andrew, this looks way awesome.
To the other commenter who was unsure what the benefit of this was over the “usual method” of reading in CSV files, I can only guess that you work on projects where you (a developer with strong knowledge of data structures and how a CSV file works) are the person who deals with the CSV files.
I’m looking at Map Fields as an excellent resource for allowing semi non technical customers to upload and safely “test” if the CSV file worked (by visual inspection of the fields in the CSV) and on that front Map Fields looks like a huge improvement in the user experience.
Thanks for taking the time to write and release this Andrew.
- Mike
This looks sweet. Current or future support for row validations, and creating an “error” file of original rows that failed the validations so that the importer can clean up the subset of failing rows and then reimport?
This looks nice. As far as providing support for row validations you may consider looking at the csv-mapper gem ( http://csv-mapper.rubyforge.org/ ) or possibly using it to handle some of the mapping and transformation logic.
@chris This very much solves a problem that exists in my problem space. We have a service that allows importing of data but with this, we don’t have to force the customer to provide their data in our format. They can basically use this to map their format to what we need.
@thanks Michael
@dr nic Nice idea, would you expect the whole import to fail and report which rows failed or to simply fail those rows, I’m leaning towards failing the entire import because that would be cleaner and easier on the uploader.
@pillowpactory Thanks, I’ll have a look