I have been working on potentially starting a small business. I spent about $5K in R&D development in 2017, but I haven't yet formed a business. If I form the business this year, is there any way to claim those expenses, either now on my 2017 return or as part of the business's 2018 tax year?

it depends; The IRS says that start-up costs are "amount paid or incurred for

Creating an active trade or business, or
Investigating the creation or acquisition of an active trade or business."You may deduct up to $5k in start-up costs in your first year in business when you actually started your biz.. This deduction is restricted if you have over $50k in start-up costs. If you have additional start-up costs over the $5k, you can amortize these costs over 15 years. Amortization means you deduct a portion of a cost every year over a period of years.You can also wait to recover your start-up costs until you sell your business or close the business, but most business owners don't want to wait that long to get the tax benefit from these start-up costs.