As the spring semester wraps up, and I basically finish the rest of my in-person classes for my college career, I wanted to look back on some of the projects I worked on. Specifically I wanted to address my fact checking project and frame generation project.
The fact checking project was arguably less important to me, even though my teacher would likely be upset to hear that. I did have fun with it though. It was a challenge to dive head first into the world of fact verification and attempt to improve upon his results. Prior to this, I had barely touched NLP as it never really stuck out to me, but this semester has given me a little more interest towards it. I primarily think that this project served to improve my understanding of transformers, as well as reinforce my basic machine learning fundamentals. Sure I also learned about some fact verification architecture and was very interested in learning more but, for the most part, I really only improved on my professor's design in minor ways. I am however, looking forward to doing more fact verification work as I head into the future. It's really just not in the books for now.
There was also the frame generation project, which isn't a project anymore. After the semester long independent study, it's officially become my thesis. I have many ideas on how I want to improve it, and my understanding of visual machine learning has increased tremendously. I started this semester off with a little dream, integrate player inputs to a frame generation pipeline to evaluate the extent that frame extrapolation methods are capable, and I accomplished that. I know it's possible, and I know I can compete with the giants. I started this semester with a small MLP network and a (frankly terrible) godot module, and I ended it with a multistage hybrid model combining CNN's, SWin transformers, and regular transformers for player input cross-attention. I prepared and presented a paper for my college's Computer Science and Engineering symposium, and I've built a committee of professor's I know I can trust and will no doubt assist me.
Looking forward, I want to build off what other's have built. I'm looking at integrating new techniques like learned optical flow and frame warping, and proposing a new architecture that built off of my last one. I'm also looking to build a test suite in Unreal Engine (matching GFFE's implementation) for a reliable benchmark for other papers to use.
