State income taxes in PA are lower than DE, but if you work in DE, you'll still owe taxes to DE, since they don't have reciprocity with PA. But you'll get a credit on your PA return for a portion of what was paid to DE, so ultimately you'll pay taxes at the higher DE rate even if you live in PA. Property taxes in PA are generally higher, but if you are renting, than maybe that doesn't matter that much. So I'm not really sure what the tax incentives are that you speak of, except for property taxes.
I live in a small Chester County PA town, right over the state line. If you want quiet, right outside of a town like mine would be ideal. You'll also spend less outside of town on housing than you would in town, at least in my town, where inventory is quite limited. You'll find rent/home prices are higher in PA than DE, but it may be worth the cost. The commute to Wilmington is not bad, maybe 20-30 minutes depending on where in Wilmington you are headed. (It is really sprawling, Wilmington.) My last job in DE was more like suburban Wilmington, and it was a 20 minute car ride, and at 10 miles one way, could have been biked if I had enough muscle for some killer hills. (I do not... yet.) I also worked in the middle of the actual city, and that was 30 minutes, so it's still (IMO) a reasonable commute. There is a lot of biking advocacy in the area, and a decent number of nice sized parks around. Biking advocacy, running groups... that kind of stuff is available in the community. There are enough cyclists for the motorists to complain about it around here. I did a lot of rides training for my first tri with the community biking group last summer; it was great. Feel free to PM for more details of my area if you want!
Someone mentioned Wilmington north around 202. 202 is fine if you don't mind your environment to be primarily strip malls. I couldn't do it.
If you have kids or are putting roots down for eventual kids, PA is hands down the place to be because of the schools. DE schools are a friggin' mess. This isn't just a "both are okay and I think mine are better" thing. I noticed when I worked in DE that anyone that had the salary to afford to live in PA did live in PA. The public schools in DE really do suck. They've tried to make it better with a bevy of charters, but it is a highly disjointed effort... meanwhile, the public schools still exist and some can barely fill their seats. Meanwhile, there are busing schemes to break up failing school districts and hide the the underperforming city schools in neighboring areas. It's awful. I have several DE friends with kids about my kids' ages, and not a single one of the families will send their kids to public DE schools. A couple families home school, a couple do charters, and the rest all pay an arm and a leg for private school. DH and I compared the cost of RE taxes in PA v. private schools in DE, and PA won by a landslide.