Can I delegate team leaders (and scheduling duties) in worshipplanning.com?

You sure can, and probably should!  Each team leader can manage their teams, including scheduling and add/removing members, and adjust notification and reminder settings (among other things).

To select a Team Leader, go to People> Teams> Select the appropriate team, and then you can drag and drop members into the "Leaders" column to assign.