If vowelized differently, the holiday name "Yom Kippurim" would spell out "Yom ke-Purim" - "A day like Purim". There is a well-known idea, with this textual basis, that Yom Kippur is a day "like Purim", with the same importance or significance as Purim.
This is hard to take at face value. Yom Kippur is the holiest day of the year, while Purim is a minor rabbinic holiday. How can the two be compared? Moreover, why is Yom Kippur compared to Purim, not Purim to Yom Kippur? Even if the comparison between Yom Kippur and Purim is valid, its order should be reversed!
Here are three possible understandings of the Ke-Purim idea. To me the first is the most meaningful.
1. Release of inhibitions
It seems clear to me that the link between Purim and Yom Kippur is the unique release of inhibitions that can occur on both. Normally, there are many legitimate actions which we do not let ourselves perform, because we worry how other people will react to them. We do not want to embarrass ourselves or get too emotional in public, so we set artificial boundaries to prevent that from happening. On only two days each year do we escape these boundaries - on Purim and on Yom Kippur.
The release of inhibitions on Purim is trivial to achieve - all you have to do is drink a lot. On Yom Kippur, it is much harder. Since this is the last chance to ask forgiveness for our misdeeds, the last opportunity to reform our actions, there is a feeling of urgency and desperation not present at any other time of year. There is a sense of "If not now, when", which can spur you to do to things you never would have contemplated on another date. Sometimes, your religious expression can become as spontaneous and natural as when you are drunk. It is not that the boundaries cease to exist, like on Purim, but that you through your efforts overcome them.
We can see an example of this in Sefer Shmuel, where Hannah prays and Eli accuses her of being drunk. Eli thought Hannah was, in a sense, celebrating Purim. But in fact, she was celebrating Yom Kippur.
This idea also explains why Yom Kippur is compared to Purim and not vice versa. The Yom Kippur experience may be more meaningful than that of Purim. But Purim is the more convenient yardstick by which to measure such experiences.
2. Acceptance of the Torah
On Shavuot, many years ago at Sinai, we accepted the Torah for the first time. On both Yom Kippur and Purim, we re-accepted it. Remember that after the Golden Calf incident, it was on Yom Kippur that Moshe returned with a new set of tablets. This day marks the second, and more permanent, acceptance of the Torah.
Purim marks a third acceptance ("kiyemu vekiblu"), which seems much more similar to the second acceptance than to the first. On Shavuot we accepted the Torah amid incredible miracles, but our acceptance lasted no longer than did the miracles. On both Yom Kippur and Purim there were no such open miracles, yet our acceptance of the Torah was much longer enduring. In this way, the historical significance of Yom Kippur and Purim is very similar.
3. Yonah and Esther
On Yom Kippur we read Sefer Yonah, in which Yonah spends "three days and three nights" in the fish's belly before delivering his message to Nineveh. Here he (presumably) does not eat or drink, and we know that he takes the opportunity to pray to God for his deliverance.
This fish episode resembles part of Megillat Esther, read on Purim. There Esther, too, meets an Eastern monarch after three days of fasting. And while not explicitly mentioned, it is very likely that she spent the three days praying to God, just as Yonah did. And Esther, like Yonah, seems hesitant to carry out her mission before finally agreeing to do so.
The Esther and Yonah stories are unusual in that in both, the story revolves around the behavior of non-Jews. In Yonah, God is principally concerned with the repentance of the Assyrians. Yonah is the instrument of this repentance, but his concern for himself, and possibly for his people (whom Assyria later conquered) are deemed irrelevant. Similarly, in the megillah, it is only the Persians who can make evil decisions, and only the Persians who can rescind them. The Jewish people will be saved somehow or other (רוח והצלה יעמוד ליהודים ממקום אחר), and if this will be accomplished through Esther's plea to Achashverosh, then Esther is just as much an instrument as Yonah was.
We, of course, focus on the "Jewish" angle of Megillat Esther. But the Persians, and not just Achashverosh and Haman, have a quite important moral role throughout the story. The residents of Shushan clearly expressed their feelings about the situation (sympathizing each time with the Jews), and many other Persians chose to either fight the Jews or convert to Judaism. It is not inconceivable that these public attitudes, particularly the pro-Jewish ones of Shushan, influenced Achashverosh's final decision to side with Esther over Haman.
How would the Jews have been saved, had not Esther managed to get the decree nullified? The answer may be that the Persian empire would lose its power to enforce decrees. Perhaps it would have been conquered like Assyria and Babylon before it. Or perhaps civil war might have broken out between the Persians and Medes (two peoples who jointly ruled the empire) or between some of the 127 diverse provinces. Either way, the capital city Shushan could have suffered mightily. As with Nineveh, Shushan through its good behavior may thereby have escaped destruction.
We read Sefer Yonah during Mincha of Yom Kippur, by which time we have had plenty of chances to repent yet our accomplishments in that regard are often meager. Perhaps the purpose is to remind ourselves not to become resigned to such a state. It is hard to imagine a group of people more physically and spiritually distant from God than the Assyrians (renowned in the ancient world for their cruelty), yet in their moment of crisis they were able to repent. We, who have been brought much closer to God, are no less capable of repentance. This message is similar to that of Megillat Esther. Even at a time of "hester panim", the leading minister of the far-off Persian empire is not beyond Divine justice, and people of Shushan - both Jewish and non-Jewish - are not beyond Divine mercy.