Troubleshooting Cython

Problem 1

In my previous post I mentioned that, “After memorizing a few common errors and dealing with them, the whole process is not really that bad”. This is true, and I see no reason why I can’t try to help with the learning process. I apologize in advance for some of the poor formatting; the WordPress publishing software absolutely slaughtered most of my word processor’s formatting and made reformatting it painful.

In chess, one way to think of a queen piece is as a combined rook and bishop. So, the way I implemented figuring moves and threat was to leverage my already written rook and bishop implementations by calling their appropriate functions. One of the first problems I ran into was with visibility; let’s take a look at how I solved the problem:

Code change (in Rook.pyx):
Original:
def _rook_is_valid_move(self, point):
Change:
cdef _rook_is_valid_move(self, point):

Problem:
Traceback (most recent call last):
File “D:\Code Library\Chess\src\main.py”, line 47, in <module>
cProfile.run(‘main()’, ‘profile.txt’)
File “C:\Python26\lib\cProfile.py”, line 29, in run
prof = prof.run(statement)
File “C:\Python26\lib\cProfile.py”, line 135, in run
return self.runctx(cmd, dict, dict)
File “C:\Python26\lib\cProfile.py”, line 140, in runctx
exec cmd in globals, locals
File “<string>”, line 1, in <module>
File “D:\Code Library\Chess\src\main.py”, line 44, in main
game_controller.main_loop()
File “D:\Code Library\Chess\src\game\game_controller.py”, line 94, in main_loop
handle_event(event)
File “D:\Code Library\Chess\src\game\game_controller.py”, line 125, in handle_event
handle_rightclick(event)
File “D:\Code Library\Chess\src\game\game_controller.py”, line 145, in handle_rightclick
handle_move(selected, boardpoint, cur_player)
File “D:\Code Library\Chess\src\game\game_controller.py”, line 150, in handle_move
if selected.is_valid_move(boardpoint):
File “queen.pyx”, line 50, in pieces.queen.Queen.is_valid_move (C:\Users\Babyface/.pyxbld\temp.win32-2.6\Release\pyrex\queen.c:713)
AttributeError: ‘module’ object has no attribute ‘_rook_is_valid_move’

Cython can’t find our newly Cython’d function. This is might be due to the fact that we haven’t cimported rook, only imported the Python version. Let’s cimport rook in queen.pyx. Alas, that does not quite fix the problem.

Problem:
Error converting Pyrex file to C:
————————————————————
…
@author: Brett Geren
”’
import piece
import bishop
cimport rook
^
————————————————————
D:\Code Library\Chess\src\pieces\queen.pyx:39:8: ‘rook.pxd’ not found

Cython can’t find rook even though it exists. If we look at the last line, we’ll see it’s because Cython is searching for a file that does not exist, the rook definition file. Cython functions (and class/extension attributes) are not externally visible by default; you can make them visible by defining them in a definition file (a C/C++ style header) [talked about here]. Let’s create a definition file that defines the function (named rook.pxd):

New Problem:
Traceback (most recent call last):
File “D:\Code Library\Chess\src\main.py”, line 41, in <module>
from game import game_controller
File “D:\Code Library\Chess\src\game\game_controller.py”, line 43, in <module>
import game_view
File “D:\Code Library\Chess\src\game\game_view.py”, line 42, in <module>
from model import game_model
File “C:\Python26\Lib\site-packages\pyximport\pyximport.py”, line 328, in load_module
self.pyxbuild_dir)
File “C:\Python26\Lib\site-packages\pyximport\pyximport.py”, line 181, in load_module
mod = imp.load_dynamic(name, so_path)
File “game_model.pyx”, line 37, in init model.game_model (C:\Users\Babyface/.pyxbld\temp.win32-2.6\Release\pyrex\game_model.c:2212)
File “C:\Python26\Lib\site-packages\pyximport\pyximport.py”, line 328, in load_module
self.pyxbuild_dir)
File “C:\Python26\Lib\site-packages\pyximport\pyximport.py”, line 181, in load_module
mod = imp.load_dynamic(name, so_path)
File “player.pyx”, line 39, in init model.player (C:\Users\Babyface/.pyxbld\temp.win32-2.6\Release\pyrex\player.c:3226)
File “C:\Python26\Lib\site-packages\pyximport\pyximport.py”, line 328, in load_module
self.pyxbuild_dir)
File “C:\Python26\Lib\site-packages\pyximport\pyximport.py”, line 181, in load_module
mod = imp.load_dynamic(name, so_path)
File “queen.pyx”, line 1, in init pieces.queen (C:\Users\Babyface/.pyxbld\temp.win32-2.6\Release\pyrex\queen.c:1363)
ImportError: Building module failed: ['ImportError: Building module failed: [\'ImportError: Building module failed: ["AttributeError: \\\'module\\\' object has no attribute \\\'__pyx_capi__\\\'\\\\n"]\\n\’]\n’]

This error is extremely unhelpful; to make matters worse, if you try and Google the problem you get nothing substantive. As awkward as this may seem, I found that the solution to this and several other “odd” problems was to simply force a recompile of the entire project by cleaning (deleting) the generated and compiled code. The location of the generated code can be found in several places in the traceback; mine is C:\Users\Babyface/.pyxbld.

Problem 2

The following illustrates a possible problem you may encounter if you try making Cython functions externally visible:

Error converting Pyrex file to C:
————————————————————
…
delta_diff = math.fabs(col_diff – row_diff)
return (delta_diff != 0 and (col_diff == 0 or row_diff == 0)
and __path_is_clear(self, point, col_diff == 0))
cdef _rook_get_possible_moves(self, list):
^
————————————————————
D:\Code Library\Chess\src\pieces\rook.pyx:81:5: Function signature does not match previous declaration

The implementation in rook.pyx:
cdef _rook_get_possible_moves(self, list)
The declaration in rook.pxd:
cdef _rook_get_possible_moves(self, list)

As the above code selections show, the function signatures sure look the same. That may be easy for an intelligent human to decipher, but what about an “unintelligent” compiler blindly following instructions? Cython can figure out what self is (since it is, in a way, pre-typed). It obviously realizes that the functions both have the same names; otherwise, we wouldn’t be getting such a specific error. This can only mean that Cython has some issue with the list argument. The list argument is a Python list and herein lays the problem; Cython thinks that the two lists have two different types. I’m not sure where I read this (I believe it was in a mailing list repo for a project that utilizes Cython), but several of the developers there were under the impression that Cython can sometimes generates several C structs and types for the same Python types if it ends up getting imported multiple times in different files. This would certainly lead to the compiler believing that the lists were of different types. The easiest way to fix this problem is to explicitly statically type list; however, list isn’t a simple C-type that we can statically type…… Thankfully, there exists a generic “object” type that I believe you can use for all Python objects. This fixes the problem:

cdef _rook_get_possible_moves(self, object list)

Problem 3

Another problem that you could face looks deceptively like the above problem:

Traceback (most recent call last):
File “D:\Code Library\Chess\src\main.py”, line 41, in <module>
from game import game_controller
File “D:\Code Library\Chess\src\game\game_controller.py”, line 43, in <module>
import game_view
File “D:\Code Library\Chess\src\game\game_view.py”, line 42, in <module>
from model import game_model
File “C:\Python26\Lib\site-packages\pyximport\pyximport.py”, line 328, in load_module
self.pyxbuild_dir)
File “C:\Python26\Lib\site-packages\pyximport\pyximport.py”, line 181, in load_module
mod = imp.load_dynamic(name, so_path)
File “game_model.pyx”, line 37, in init model.game_model (C:\Users\Babyface/.pyxbld\temp.win32-2.6\Release\pyrex\game_model.c:2212)
File “C:\Python26\Lib\site-packages\pyximport\pyximport.py”, line 328, in load_module
self.pyxbuild_dir)
File “C:\Python26\Lib\site-packages\pyximport\pyximport.py”, line 181, in load_module
mod = imp.load_dynamic(name, so_path)
File “player.pyx”, line 39, in init model.player (C:\Users\Babyface/.pyxbld\temp.win32-2.6\Release\pyrex\player.c:3226)
File “C:\Python26\Lib\site-packages\pyximport\pyximport.py”, line 328, in load_module
self.pyxbuild_dir)
File “C:\Python26\Lib\site-packages\pyximport\pyximport.py”, line 181, in load_module
mod = imp.load_dynamic(name, so_path)
File “queen.pyx”, line 1, in init pieces.queen (C:\Users\Babyface/.pyxbld\temp.win32-2.6\Release\pyrex\queen.c:1267)
ImportError: Building module failed: ['ImportError: Building module failed: ["ImportError: Building module failed: [\'TypeError: C function pieces.bishop._bishop_get_possible_moves has wrong signature (expected int (PyObject *, PyObject *), got PyObject *(PyObject *, PyObject *))\\\\n\']\\n”]\n’]

While this looks like another issue of the compiler incorrectly figuring types, you can see that the incorrectly figured type is the return value’s type. I found that statically typing the return value did NOT fix this issue; however, cleaning and recompiling did.

Problem 4

The next problem may be trivial, but a late night of coding ended up causing me to waste something like twenty minutes on the below:

Error converting Pyrex file to C:
————————————————————
…
#rook meets requirements
cdef int rook_col = (LEFT_ROOK_COL if
point[0] == LEFT_CASTLE_KING_POS
else RIGHT_ROOK_COL)
Rook rook = self.owner.game_model.piece_for_boardpoint((rook_col, valid_row))
^
————————————————————
D:\Code Library\Chess\src\pieces\king.pyx:129:13: Syntax error in simple statement list

The problem is that you forgot to tell the compiler that you are about to type the variable (you left out the cdef keyword).

Problem 5

The next problem sort of reiterates a lesson learned earlier:

AttributeError: ‘pieces.king.King’ object has no attribute ‘is_in_check’

Calling source code (in player.pyx):
if self.king.is_in_check(self.king.point):
ret = False

Source function in question (in king.pxd and king.pyx):
cdef is_in_check(King self, point)

Once again, the function exists but Cython can’t find it. Unlike above, the function is defined in a definition file, making it publically available to other modules. One lesson we learned earlier is that explicitly typing can solve a lot of your problems, so let’s try that here:

cdef King king = self.king
if king.is_in_check(self.king.point):
ret = False

The above will fix the issue since Cython will type king and do the correct lookup for the function. The actual issue, more than likely, was that the calling code was searching in “Python space” and it had trouble resolving the search for the function in “Cython space” (where it actually existed). Another method to fix the problem was to simply avoid OOP entirely (which is probably the source of the search issues) and explicitly call King’s is_in_check function. This will of course work since the function has been made publically available:

if King.is_in_check(self.king, self.king.point):
ret = False

Above are five problems that I personally found either frustrating to solve or common enough to make them worth writing about. Hopefully, getting over the initial Cython learning curve is slightly less painful now.

Cython

Sometimes, you do not want the computer to beat the snot out of you while playing a game, say chess. In fact, if you are like me then you are still quite a novice and would appreciate competitive game play from a constantly available source, not a massacre. This is exactly the point behind my newest project, an adaptive chess engine.

The point of the project is to develop a chess AI that will improve with you and still let you win some. Other projects of this nature have been attempted before, but as far as I can tell, not that much academic work has gone into making a more competitive chess AI. If you know even a little about AI, then you probably have at least heard of Alpha-Beta/Minimax/Negamax search. These are general algorithms to generate solutions to two-person competitive games such as chess, checkers, go, etc. These algorithms make the assumption of perfect play by both players (not really a weakness of the algorithms) and that both players are working under a well-known strategy. These algorithms then “look ahead” by playing several moves in advance by playing [almost] every possible move in a turn and then picking the best one. Repeating this should generate a Nash equilibrium and give you the “best” possible move that your opponent (who plays perfectly) will allow.

One obvious problem with this process is that you have to evaluate a lot of possible moves and board positions, roughly avgMoves^depth. This means that Alpha-Beta search grows exponentially with the depth, which can cause alpha-beta searches to take some time. This is especially true for my original, naïve Python implementation. My initial implementation was in fact too slow to allow enough look-ahead to accomplish any real competitive play.

One possible solution to this problem is to lose some features of Python and program in C through the use of a 3^rd party compiler called Cython, a language that forms pretty much a superset of Python. Cython allows the programmer to decide what gets programmed in fast, efficient C/C++ or slower, more flexible Python. It is actually even possible to tell Cython to do both and then use the C/C++ version whenever possible. Much of the improvements from Cython involve getting rid of the performance hit from interpretation: variables can be typed, function can be typed, functions can be marked to be completely compiled in C/C++, and you can write classes (Extensions) in C and then use them in both Python and C code.

The first thing I noticed about Cython was just how easy it was to install, setup, and use. Provided you can get a C/C++ compiler installed that you like (installing MinGW on Windows can be a hassle if you don’t know that they have a “secret” auto-installer found here; it used to be REALLY hard to find), the entire process leading up to actually using Cython takes just a few minutes, including download. After that, you can start compiling their examples in Cython, which went off without a hitch. I then decided to jump-in and start improving my code.

The first order of business was to integrate Cython compilation and use into my already developed codebase. The approach I almost took, and the one that most resembles actually programming in C/C++, is to write, compile, run. As one of the Cython developers put it, “one of the deep insights in Python’s implementation is that a language can be compiled (Python modules are compiled to .pyc) files and hide that compilation process from the end-user so that they do not have to worry about it. Pyximport does this for Cython modules.” [Statement found here] The tutorial goes on how to describe how to use Pyximport and how you can include it in your sitecustomize.py, but never mentions that if you only have 1 or a handful of entry points; you can just put the includes there and it appears to work just fine [More compilation methods here].

After I had on-the-fly compilation of both Python and Cython code handled, I decided to try adding some simple static typing to a handful of files. Low and behold, after only renaming the file extensions to .pyx and then adding some cdef’s I had Cython code being compiled into and running as C code. For “bleeding edge” technology (.13 was just released), Cython sure is easy to use. After that, I decided I would convert my commonly used Python “constants” to real C/C++ constants for some free performance. Cython does not really support constants yet; however, they suggest using anonymous enum types for this purpose [seen here]. One method I almost used (which actually is not a “constant” and still appears to take Python interpreter hits) is the use of DEF statements [seen here] which act just like #include’s in C/C++ except that they appear to then be converted to Python types.

After the conversion to “real” C/C++ constants, I ran into my first problems. The first problem I had to tackle was that I had trouble interpreting the error message given by my other problems. Cython has multiple steps in the conversion from your Cython/Python into a compiled shared object or executable. Any one of these steps can crash, give an error, etc. Learning to read and interpret these different error messages is possibly some of the hardest stuff when working with Cython. I found the easiest way to debug the compilation errors was to make epsilon changes to my code and then try to run it. This made figuring out the source and solution to the problem considerably easier. After memorizing a few common errors and dealing with them, the whole process is not really that bad.

After learning to debug compilation errors, I eventually learned that code in Cython .pyx files is not made publically available to other files normally. There are varieties of ways to fix this, the easiest in this case was to make a definition file, basically a C/C++ header [talked about here]. However, even after this I still had issues; it turns out that the importation of Cython code requires the use of a cimport statement, imports for Cython code [This and visibility better explained here].

After learning to cope with compilation errors, I then proceeded to cdef heavily hit sections of code and eventually decided to tackle extensions [seen here]. Cython lets you create new built-in types through the use of “Extensions”; these extensions are basically Python classes that are almost completely C types and methods. These extensions can dramatically improve performance over classes if the code is heavily used, especially in already written Cython code since you can avoid using Python methods, type conversion, wrapping, etc. To be honest, I found writing and using extensions to be a bit tedious; this is definitely an area where you want to do continual compilation to look for errors. One item that the extensions page does NOT explicitly cover is that there is support for an “object” type that, as far as I can tell, allows you to pass any general Python object. This was endlessly useful when converting some of my Python classes to Cython since many of them were extensions of other classes I had written or were mutually dependent. The object type allowed me to type these other, currently Python, classes and test the current extension until I could write and test the others. Later, I can go back and properly type the “objects” [object type explanation here].

This concludes my first excursion into Cython. I certainly did not come close to exploring all that Cython can be used for, but I feel that I covered some necessary basics. In particular, one of the coolest things Cython allows you to do is write or use handwritten C/C++ code and then provide a Python interface through Cython code. Overall, I would have to say that I am thoroughly impressed by Cython, especially its seeming stability and maturity given such a young project.

TinyPack

Introduction

In my last post I stated that I participated in the 2010 Missouri S&T REU, but I didn’t state what my specific project was, TinyPack. TinyPack is a wireless sensor network compression algorithm that is being worked on by Thomas Szalapski, a Missouri S&T grad student. My project is to implement TinyPack in a simulation and compare the energy consumption results with a similarly themed wireless sensor network compression algorithms, such as SLZW.

Wireless sensor network compression algorithms compress at the nodes and thus each individual node need not send nearly as much data as they would otherwise. This is important because the sensors have a drastically disproportionate cost between data transmission and general processing, which is illustrated below. We can see the energy cost to send one byte of data for various radios.

TinyPack

TinyPack in particular uses a Delta Huffman encoding scheme, which breaks the data apart from generally large, but similarly ranged values, into small, delta values stemming at zero from a median value taken from the data set. Then, a Huffman diagram (with seed) and Huffman encoded values are transmitted for the future reconstruction of the data. Ideally, this will result in significantly reduced energy consumption. Now let’s go over some quick background information about the simulation.

Our simulation is written to run on TinyOS, a free, open source operating system that as a “minimal subset” of OS features implemented. TinyOS is targeted to wireless sensor networks (WSNs) due to its size, ease of use for low-level tasks, and its component-based design. TinyOS is completely written in nesC, a derivative of C, which has been modified to handle the memory constraints of WSN’s. NesC is unique in that it has a single call stack so all threads/processes work as “cooperative tasks”. Some NesC items are preemptable while most are not. An interesting bit about TinyOS is that there is not, by default, a heap; therefore, NesC doesn’t natively support malloc, free, etc.

TinyOS has a simulation framework called TOSSIM that allows you to run your same code in a simulation on a PC. In addition, an extension to TOSSIM was made called PowerTOSSIM that allows for the gathering of power consumption data. With PowerTOSSIM we have developed tests and measures to find the potential advantages of our algorithm over others. Now that at least the basics have been covered, let’s look at the simulation results.

Result Analysis

It is clear from the first chart that Delta-Huffman Compression (DHC) does an excellent job at compressing our dataset. SLZW also does an acceptable job at compressing and saving space. Uncompressed, of course, saves no space. The space savings are inversely correlated to power consumption; smaller packets take less energy to send. It is obvious then that Delta-Huffman Compression would far outpace the others in power savings. This is only for one dataset, but, so far it appears that Delta-Huffman Compression is definitely a contender when it comes to wireless sensor network compression algorithms.

Future Work

As stated above, so far we have only tested Delta-Huffman Compression with one dataset. The most obvious course of action for future work would be to gather results for other datasets. Certainly, algorithm performance will vary from dataset to dataset. Learning Delta-Huffman Compression’s weakness with datasets would be useful for tweaking and improving the algorithm or in the very least illustrating with what datasets it should not be used. Finally, to truly validate our research and results we would need to actually run our code on sensor motes.

Missouri S&T REU

This summer, I participated in a research and training experience for undergraduates (REU) at the Missouri University of Science and Technology. REU’s are summer research positions funded by the National Science Foundation (NSF) and hosted by universities across the country; REU’s typically focus on preparing undergrads for research in a topic or field.

My particular REU involved working with the departments of electrical and computer engineering and focused on researching, improving, securing, and utilizing wireless sensor computing techniques. This obviously means that I learned and employed skills and techniques that do not fall in my chosen major, computer science. To be more specific, our REU worked with hardware (proprietary wireless sensors) to improve communication techniques, develop schemas for optimal sensor network layout, etc. These tasks involve experiences and interactions with hardware, experiences that my current discipline generally doesn’t explore.

Mouseless Updates

I’ve decided upon the name “Mouseless” for my mouse-control-through-hand project. I’ve done quite a bit of work on the project, but I never found the time (or maybe the will ;P) to post anything until now. The project works relatively well as of right now. Isolating the hand in the image is quite successful given one requirement, while gesture recognition is decently successful given a few requirements.

The hand requirement is that you use some sort of solid-colored glove whose color is distinct from the background. This allows Mouseless to do hand isolation through color comparison instead of some sort of geometric algorithm. I decided to go with the color comparison route because of its relative ease to code, its effectiveness, and the fact that everyone, at heart, wants to wear a single glove like Micheal Jackson (let’s all do a silent moonwalk in remembrance).

Gesture recognition has a few issues. All of these issues are due to how the recognition is done, through a self organizing map. The first issue is that the user has to “train” it’s computer on the gestures they want to use. The second issue is that each time the user changes the color of their glove they have to re-train the computer since the color is stored in the SOM (more on this later). The next issue is that slight rotations in gestures cause a lot of “noise” in the SOM around the gesture. This causes the SOM to store the noisy data which is a little problematic for recognition. One rather nice thing about the use of the SOM is that the user can choose their own gestures. Since the gestures can be trained, there is no need to require that any action be tied to a certain gesture. The other nice thing about the SOM is that, despite all of its problems, it still does a pretty decent job at recognition.

One idea I’ve had recently (but haven’t had a chance to implement) is to convert the image to gray-scale (or maybe even to all black right) before it goes into the SOM. This would mean that the user can change glove colors without any re-training necessary. This might also help a little with the noise from gesture rotation. Anyway, I’m sure you are all chomping at the bit to see some screenshots, so here they are:

(Please excuse the stitches on my left cheek; I recently had surgery. Hopefully, the stylish purple glove will draw your focus away from it .)

Mice are so 14th-Century Black Plague Anyway

I’m taking Artificial Intelligence this semester and one of the course requirements is to design and do a “final project”. The final project can take the form of an improvement upon a previous assignment in the class, or it can be an adventure into a completely new area. While thinking about various project ideas, I came upon a project that fit into the latter. I am going to turn the user’s hand into a mouse!

More specifically, I was going to write software that takes images from a webcam and uses the images, the user’s hand movements, and the user’s hand gestures to act as the mouse. As of now, I’m thinking that the hand moving will move the mouse, a closed hand (a fist) will be a single left-click, and so a closed fist with a moving hand will be a drag. I’m still considering various ideas for right buttons, scroll wheels, and possibly thumb buttons.

Google Summer of Code Project Reflection

First off, sorry about the pretty plain title . As you have probably figured out this entry is a reflection about my GSoC project, my work, etc. While reflecting about this summer one of the first things I realized was just how much I learned. I was actually glad I did the Odyssey if for no other reason than to realize how useful this summer really was. My experiences this summer allowed me to learn quite a lot as a programmer and to figure out what I despise and what I love as a person.

My most obvious learning experience, which also happened to be the one Google wanted me to have the most, was learning to adapt to open source project dynamic. It was a completely new experience to do all work, consultation, meetings, etc hundreds of miles away from the other project members. Initially, I wanted to do all consultation and progress reports solely through my mentor, in a style with what I was more familiar with from college. My mentor never expressed any problems with this, but I eventually figured out that doing all consultation, progress reports, etc on the developers mailing list not only caused me to receive answers quicker, but also caused the general quality of answers to increase as several members put in their input and answers were revised. Now, this revelation may have seemed obvious, but as I said, it was unfamiliar territory for me. Part of the reason for my initial reluctance to use the mailing list was because I did not want to admit to my “bosses” that I had questions/issues. Who wants to admit a mistake or problem in front of a group of their peers and employers? After I became comfortable with sharing on the developers mailing list, I noticed a considerable increase in productivity, a higher level of quality of code, and it eventually led to me actually feeling like a member of the “SIP Communicator developers family”.

Besides becoming familiar with and using the many development tools, I learned huge amounts about how issues in large programs are handled. In a previous entry, I mentioned Felix; I had always wondered how plug-ins and such are handled, and I now have at least an idea of one approach. One “silly” thing I am particularly glad to have learned, was to learn to get used to having opening brackets indented on new lines. In the past, I have always had the option to write code exactly how I want. This favoring of my personal tendencies actually ended up making it hard for me to read code not written in the same manner. After an entire summer of being forced to have the opening bracket on a new line, I feel I am proficient in reading both code styles.

On a personal note, this summer actually proved to be quite useful for learning about my personal likes and dislikes. My parents moved about a month before college started and I don’t go home often  so when this summer rolled around and I worked from my house I had a rather hard time finding ways to occupy my free time. I had never considered loneliness an issue to consider when picking where to work. In the past had you asked me if I could work from my house I would have said I would LOVE to do that; however, I would currently add the caveat that I would have to live somewhere where I was familiar with lots of people. Another thing I learned, which is also possibly more useful, is that I thoroughly enjoy software development for the general public (versus proprietary software that probably only gets used by a very small number of people). I enjoy seeing and hearing other people use my work. It is rather nice to know at least one avenue I can pursue when I am searching for a job. I hope to do an REU this summer so I can get an idea of whether I like research or software development more. I know that I also thoroughly enjoyed my work last summer. By the time I graduate I hope to have had enough work experience to have a better idea of which particular type of work I enjoy more.