Fixing Account Sync Errors in Personal Capital

I love collecting data about my life. I track the things I get done, the foods I eat, the dollars I spend, the books I read, and my net worth.

I recently switched from Mint.com (better for tracking spending) to Personal Capital (better for tracking investments) to more effectively track my investments and make better decisions for the long term.

When I was making the move to Personal Capital, I found that some of my accounts weren't syncing properly - Personal Capital said the username/password I was supplying was incorrect. I knew this couldn't be the case since (1) I just logged into the account successfully outside of Personal Capital and (2) I use a password manager and was copy/pasting directly into Personal Capital.

Searching Google offered some help, but not much. I found other people having trouble connecting their accounts like me, but nothing resolved (folks, this post is for you). Even so, I figured the problem had to be on my end, since these were pretty significant financial institutions I was using and I thought there was no way syncing was broken for everyone.

After trying a few different things, I settled on my password manager. I've had troubles in the past with various websites balking at the length or choice of special-characters 1Password picked for me.

I changed my password on one of the broken account links to a strict alphanumeric password under 15 characters (something like Lhfey982DP7Z2FF) and everything worked.

I played around with the configuration some more and from what I can tell, Personal Capital is stripping out characters it doesn't like from your password, and sending the cleaned password to your financial institution with characters missing - which will make your password incorrect.

From what I can tell, here are the characters not allowed in your password that Personal Capital won't warn you about. The list may be a bit different, but this is probably a safe place to start: >, <, /, \, ], [.

If you are experiencing the same problem, change your password to something without the characters above and try again. Hopefully, everything will work out for you.

In retrospect, a warning on Personal Capital's side would have been nice, but all my accounts are syncing properly now, so I'm over it.

Be The Worst

When I was in college, I took a particularly challenging Probability and Statistics class - most of the subject matter was new to me and it took me all of about five minutes to realize I was severely out of my element. One day after class, I expressed some frustration because I felt like there was only so much I could learn from the book and online examples, and that limited knowledge alone was insufficient to excel in my class. Overhearing my complaint, my professor gave me some great advice that I will never forget: "All you have to do to succeed in this class, or any area of life, is to find a group of people who are smarter than you and start working with them."

What do I do when my rate of learning has leveled off?

You have taken the time to learn something new, immersed yourself in the subject matter, practiced diligently for a period of time and look up and see that you are ahead of those around you. Congratulations on your accomplishment. Now what? There are only so many books, blogs, and other resources you can learn from until you reach a point where you must learn new skills from other humans. We are social creatures, by nature. We rely on each other to survive and part of that survival involves learning from each other. It's difficult to imagine a world where we all had to learn everything on our own and couldn't benefit from the knowledge and insight of those around us.

In Apprenticeship Patterns, Dave Hoover and Adewale Oshineye outline this very problem and propose a solution [pattern] called Be the Worst.

The idea behind being the worst is to surround yourself with people who are better than you in a given area - this will force you into a position where you have room to grow. Be the worst one around at programming or guitar or soccer. Over time, the better people around you will cause you to elevate your game, mostly through observation and instruction.

[As] the weakest member of the team, you should be working harder than anyone else. This is because the goal is not to stay the weakest, but to start at the bottom and work your way up. You do this by consciously finding ways to improve and mimicking the stronger [team members] until you are on the same level as the rest of the team. -Apprenticeship Patterns

Of course, there are risks associated with this strategy. You will, by definition, be the least effective person around. This means that your contributions will result in a net drag on the team. Because of this, most teams and organizations won't put up with you for very long. You will also force yourself into a sink-or-swim situation with some very real consequences if you don't improve quickly. The best solution to mitigate this risk is to outwork the problem - make being the worst a temporary solution to solve a larger problem. Focus on consistent, determined effort to get you from where you are to where you want to be. If you find yourself getting distressed, you may have bitten off too much and it's time to scale back a bit.

Being the worst is a great way to improve your effectiveness in a given subject area quickly. However, this is a risky strategy that takes concerted effort to constantly improve and ensure you aren't drowning in the more intense sink-or-swim environment. Next time you are looking for a way to improve your skills and have exhausted all of the resources currently available, seek out a change in scenery. Find a team filled with people who are better than you, work hard, and watch yourself grow at a rate you didn't think was possible.

CAP Theorem: Explained

Several years ago, building performance into a software system was simple - you either increased your hardware resources (scale up) or modified your application to run more efficiently (performance tuning). Today, there's a third option: horizontal scaling (scale out).

Horizontal scaling of software systems has become necessary in recent years, due to the global nature of computing and the ever-increasing performance demands on applications. In many cases, it is no longer acceptable to run a single server with a single database in a single data center adjacent to your company's headquarters. We need truly distributed environments to tackle the business challenges of today.

Unfortunately, the performance benefits that horizontal scaling provides come at a cost - complexity. Distributed systems introduce many more factors into the performance equation than existed before. Data records vary across clients/nodes in different locations. Single points of failure destroy system up-time, and intermittent network issues creep up at the worst possible time.

These concerns of consistency (C), availability (A), and partition tolerance (P) across distributed systems make up what Eric Brewer coined as the CAP Theorem. Simply put, the CAP theorem demonstrates that any distributed system cannot guaranty C, A, and P simultaneously, rather, trade-offs must be made at a point-in-time to achieve the level of performance and availability required for a specific task.

CAP Theorem Trade-offs

[C] Consistency - All nodes see the same data at the same time.

Simply put, performing a read operation will return the value of the most recent write operation causing all nodes to return the same data. A system has consistency if a transaction starts with the system in a consistent state, and ends with the system in a consistent state. In this model, a system can (and does) shift into an inconsistent state during a transaction, but the entire transaction gets rolled back if there is an error during any stage in the process.

Typical relational databases are consistent: SQL Server, MySQL, and PostgreSQL.

[A] Availability - Every request gets a response on success/failure.

Achieving availability in a distributed system requires that the system remains operational 100% of the time. Every client gets a response, regardless of the state of any individual node in the system. This metric is trivial to measure: either you can submit read/write commands, or you cannot.

Typical relational databases are also available: SQL Server, MySQL, and PostgreSQL. This means that relational databases exist in the CA space - consistency and availability. However, CA is not only reserved for relational databases - some document-oriented tools like ElasticSearch also fall under the CA umbrella.

[P] Partition Tolerance - System continues to work despite message loss or partial failure.

Most people think of their data store as a single node in the network. "This is our production SQL Server instance". Anyone who has run a production instance for more than four minutes, quickly realizes that this creates a single point of failure. A system that is partition-tolerant can sustain any amount of network failure that doesn't result in a failure of the entire network. Data records are sufficiently replicated across combinations of nodes and networks to keep the system up through intermittent outages.

Storage systems that fall under Partition Tolerance with Consistency (CP): MongoDB, Redis, AppFabric Caching, and MemcacheDB. CP systems make for excellent distributed caches since every client gets the same data, and the system is partitioned across network boundaries.

Storage systems that fall under Partition Tolerance with Availability (AP) include DynamoDB, CouchDB, and Cassandra.

Conclusion

Distributed systems allow us to achieve a level of computing power and availability that were simply not available in yesteryears. Our systems have higher performance, lower latency, and near 100% up-time in data centers that span the entire globe. Best of all, the systems of today are run on commodity hardware that is easily obtainable and configurable with costs approaching $0.

All of this computing power and benefit comes at a price, however. Distributed systems are more complex than their single-network counterparts. There are many more tools and skills that need to be acquired in order to create a truly scalable, high performance system. Understanding the complexity incurred in distributed systems, making the appropriate trade-offs for the task at hand (CAP), and selecting the right tool for the job are all critical skills in a world where computing systems are moving out, not up.