Jump Start or False Start? A Theoretical and Empirical Evaluation of LLM-initialized Bandits | ScienceToStartup