Convolution for NLP —Temporal Convolution
(Convolutional Neural Networks for Sentence Classification)
Fully connected Advantage layer link
CPU to GPU
local input = torch.Tensor(32,2):uniform() input = input:cuda() local output = model:forward(input)
… or create them directly as
local input = torch.CudaTensor(32,2):uniform() local output = model:forward(input)
Using a GPU in Torch
Using a GPU in Torch is incredibly easy. Getting set up is simply a matter of requiring the
cutorchpackage and using the CudaTensor type for your tensors.
cutorch = require 'cutorch' x = torch.CudaTensor(2,2):uniform(-1,1)
Now all of the operations that involve
x will computed on the GPU.
If you have a tensor that is not a
CudaTensor but want to make it one, you can use the
cuda()function to return a
CudaTensor copy of the original:
x = torch.Tensor(2,2):zero() xCuda = x:cuda()
You can see what type of tensor you have by inspecting it in the console:
th> x 0 0 0 0 [torch.DoubleTensor of size 2x2] th> xCuda 0 0 0 0 [torch.CudaTensor of size 2x2]
You can also convert back to a CPU tensor:
th> y = xCuda:double() th> y 0 0 0 0 [torch.DoubleTensor of size 2x2]
Keep in mind that the parameter matrices of the
nn.Module objects also need to be configured for GPU use, as these contain internal tensors for storing parameters, and the forward/backward propagation state.
Lucky for us, these also have
linearMap = nn.Linear(M,M):cuda()
Thoughts on torch
Basically the only hard thing I need to do when developing with torch is thinking about tensor dimensions. It seems an inordinate amount of my brain cycles are consumed in this way. But I don’t fault torch at this, as I think it’s an unavoidable aspect of working with multi-dimensional tensors.
Lua also has many great things going for it, and by proxy these are also reasons why torch is great:
- Very fast execution time (very little reason to consider C++ or other compiled languages).
- Can be easily embedded in other applications.
- Nice profiler provided by LuaJIT.
- Given it’s interpreted, interactively prototyping code makes it easy to explore how things work.
Unfortunately, there are a number of not so fun aspects of lua:
- Feels primitive and very bare-bones compared to other scripting languages.
- Rather unhelpful stack traces.
- Debugging facilities seem rather lacking.
- nil. Because variables don’t need to be defined, spelling mistakes and other minor errors result in nil, which combined with poor stack traces sometimes makes it hard to locate the problem.
Show Line Numbers in Jyputer Notebook
shortcut key: L to toggle line number
Hierarchical Data Format (HDF) technologies uses to management of large and complex data collections and ensure long-term access to HDF data.
HDF5 is a data model, library, and file format for storing and managing data. It supports an unlimited variety of datatypes, and is designed for flexible and efficient I/O and for high volume and complex data. HDF5 is portable and is extensible, allowing applications to evolve in their use of HDF5. The HDF5 Technology suite includes tools and applications for managing, manipulating, viewing, and analyzing data in the HDF5 format.
The h5py package is a Pythonic interface to the HDF5 binary data format.
It lets you store huge amounts of numerical data, and easily manipulate that data from NumPy. For example, you can slice into multi-terabyte datasets stored on disk, as if they were real NumPy arrays. Thousands of datasets can be stored in a single file, categorized and tagged however you want.
You can use python-h5py.
sudo apt-get install python-h5py
And then in your Python file try:
Vision and NLP Conf
- CVPR: IEEE Conference on Computer Vision and Pattern Recognition
- ICCV: International Conference on Computer Vision
- ECCV: European Conference on Computer Vision
- NIPS: Neural Information Processing Systems
- ICLR: International Conference on Learning Representations
- ICML: International Conference on Machine Learning
- EMNLP: Empirical Methods in Natural Language Processing
- ACL: Association for Computational Linguistics
- NAACL: The North American Chapter of the Association for Computational Linguistics
- ACCV: Asian Conference on Computer Vision
- IJCV: International Journal of Computer Vision
Past CS229 Projects: Example projects from Stanford machine learning class