Consensus seems to be that this series improves with each book, but I'm not entirely sure that I agree. This series changes with each book, but I can see how someone might prefer book 1 to book 3. It's all a matter of priorities. Book 1, Shades of Milk and Honey, is much smaller in scope. It's an intimate portrait of a couple and of the few closest friends, family members, and neighbors who are around while they are falling in love. The comparisons to Jane Austen abound, and are at least somewhat sensible because the book is such a slowly-paced study of quiet moments and details in Regency society. Of course, there's also some magic, and the magic is delightful. My primary complaint about Shades is that there wasn't quite enough magic to make the book firmly and unquestionably more than an Austen pretender. I would have liked to have at least twice as much description of glamour and artistic collaboration than there was.
Then we move along to book 3, Without a Summer, and I think the comparisons with Jane Austen are now patently ridiculous. While book 1 was magic-flavored Austen, book 3 is very slightly Austen-flavored historical romantic fantasy, with an emphasis on history. Austen never broadened her scope to politics, never wrote about espionage, never had a main character spend time in jail, be tortured, or tried for treason. Austen never took on the troubling issues of workers' rights and compensation in a changing economy. And she never depicted a character who was as disgustingly, frighteningly evil as Vincent's father, Lord Verbury. The facts that this book is set in alt-Regency England and one of the female characters often begins her statements with "La!" are about the only Austen-esque characteristics of this book. So stop with that nonsense.
I can see that some people think this book is the best in the series for the reasons that I've alluded to above -- it's so much more ambitious than Shades, and the writing is intelligent, confident, and clearly aware of a much bigger picture that Kowal can draw and draw from as she chooses. It's nice to see a depiction of a relationship continue to succeed over the course of a series when the characters have been wed since book 1 and in the hands of many other authors would become boring after that. Jane and Vincent are interesting just being together -- they don't have to be chasing and angsting to be worthy leads. Happily, Melody makes a surprisingly endearing counterpoint to Jane as a lead in this book, as well. She turns out to have quite a lot of depth and some surprises up her sleeves.
I really loved the first half of this book -- I felt like I was getting more magic, like I'd wanted, and there was a brief passage where Vincent essentially makes a modern art glamour that really moved and excited me. If Kowal had gone in that direction even more, I'd have been in raptures.
It's just a matter of personal taste that I wasn't thrilled with the addition of Vincent's loathsome father. He makes a very powerful villain, but I find it upsetting to read such detail of cruelty within a family. The only objective problem that I have with the plot involving Lord Verbury is that if he is so awful and Vincent knew by the age of 20 that he needed to get away from his father and have no contact whatsoever, he allows a few too many incidents of contact between his father and himself and Jane in this book. It diminishes Vincent in a way that alienated me when he allowed his father to hurt Jane more than once.
Another thing that I feel is a weakness of this book, or at least a missed opportunity, is how not instrumental glamour is in achieving any resolution to the story arc with the protest, betrayal, jailing, and trial. Not only do I want to see characters with magic use their magic to deal with their unique life circumstances, I do not like a book to feature magic prominently in the first 200 pages, and then barely mention it in the last 200.
Mary Robinette Kowal still hasn't written the book that I, personally, hope she will write -- a book with the more intimate tone and smaller scope of her first book, with more ambitious descriptions of and uses for magic. I think Shades of Milk and Honey was less subject to criticism because it was a less ambitious book, and succeeded almost entirely at what it was trying to do. Without a Summer is much more ambitious and often succeeds at what it's trying to do... but I wished it tried to do some different things.
Read it if you like Jane and Vincent and enjoyed book 1, but prepare to be shocked by the Luddites and Vincent's family and the jail and the trial and the Irish-Catholic-hating... We're not in Austenland, anymore!