Qualitative Research: Mobile Investing Apps
Skills & Tools
About
As part of my qualitative research methods course, students were asked to pair up and conduct a small-scale, generative qualitative research project. My project partner and I proposed conducting a qualitative research study to assess users' experiences with mobile investment platforms in order to identify potential areas for improvement and gain an understanding of the existing features that users think work well.
Duration
Spring 2023 - 14 weeks
My Contributions
After deciding on a research topic area, I helped conduct a literature review to examine existing, prior research within our topic area and identify any research knowledge gaps. During the research planning phase, I put together a scoped research plan, outlined participant recruitment and data collection strategies, and crafted an interview guide and survey. During the research phase, I facilitated the distribution of our survey and interviewed prospective users, which yielded rich, qualitative data. I helped analyze the user research data by conducting a thematic analysis and coding survey responses and interview transcripts. I then identified themes and key findings from our research.
Team
Grace M.
Ahona B.
Overview
Background
Mobile investment platforms are used by millions of consumers [9] and have the capacity to positively impact users’ financial well-being, while improving financial literacy and investment decision-making abilities when users have limited time, knowledge, expertise, and resources.
We set out to conduct a qualitative research study to assess users' experiences with mobile investment platforms in order to identify potential areas for improvement and gain an understanding of the existing features that work well. By better understanding user pain points and exploring opportunities to improve the mobile investing experience, we aimed to identify a set of recommendations for mobile investment platforms to better meet the needs of users.
Literature Review
What is known already?
In order to identify our central research question and understand the scope of the existing research within our research topic area, we conducted a literature review to examine relevant published HCI research. By conducting a literature review, we were able to identify what prior research had been done related to our project focus, what were the main findings, and what relevant theories existed in the problem space.
From our literature review, we found a few research findings to be particularly valuable:
- A 2020 study conducted on adoption of mobile investment technologies found that investment knowledge, experience using mobile banking or payment applications, and ownership of certain investment vehicles were strong determinants of mobile investment decision-making [5]
- A 2021 study conducted on design patterns of investing applications, the differences across platforms, and their effects on investing behaviors introduced a set of design guidelines for encouraging healthy investing and trading behaviors [9]
- A 2021 research paper explored the criticisms of Robinhood, a relatively new and controversial mobile investing application, and advocated for uninformed retail investors to seek investment services elsewhere in order to avoid investment harm [3]
Problem
Knowledge gap
Although research had been conducted on the adoption of mobile investing applications, the design patterns of these applications, the differences across platforms, investing behaviors and biases of users, and criticisms of mobile investing applications, we suggested that we lacked an in-depth understanding of users experiences with mobile investing applications and how well existing platforms are addressing users’ needs and pain points.
Research Questions and Purpose
Central Research Question
Research Opportunity
What's the value?
By assessing users' experiences with mobile investment platforms, better understanding user pain points, and exploring opportunities to improve the mobile investing experience, we planned to identify a set of recommendations for mobile investment platforms to better meet the needs of users in order to improve the overall experience of using mobile investing applications.
Detailed Research Questions
To further define the focus of what we planned to research and investigate, we came up with two detailed research questions, which helped guide us through our research:
Scoped Research Plan
Survey
Research method
Before conducting interviews, we distributed a survey in order to collect responses that might identify insights we would want to explore further during interviews and utilize responses to recruit potential interview participants. The Qualtrics survey we crafted was made up of 15 total questions – 11 multiple choice questions, 3 open-ended questions, and 1 question about interview participation. It was distributed to Ivy College of Business students and we received 27 responses.
The questions, which you can find a sample of below, were structured to be a mix of open-ended qualitative questions and quantitative questions and focused on exploring the aspects and features of mobile investment applications that users think work well, while also blending in some questions to identify user pain points that could be discussed in more depth during interviews.
Interviews
Research method
Following the completion of the survey, we conducted five remote, semi-structured interviews, which each lasted between 30-45 minutes. Interview participants were aged 19-31 and ranged from avid to less experienced retail investors. We recruited interview participants based on target user criteria: individuals with experience using mobile investing applications.
Goal
During interviews, we wanted to get to know participants, obtain an understanding of their backgrounds, ask them about their prior experiences using mobile investing applications, and inquire about their pain points and any needs that were not being addressed. We wanted to develop a sense of any gaps in the capabilities of existing mobile investing platforms and find out what aspects of current platforms could be improved, while also identifying which aspects and features of mobile investment applications users think work well.
Research Analysis
Through our research process, we were able to collect rich, qualitative data and developed a deeper understanding about users' experiences with mobile investment platforms. To analyze this data, we used thematic analysis.
Coding
Interview transcripts were coded bottom-up using Taguette, while survey responses were coded bottom-up using comments in an aggregated survey response spreadsheet.
Our collaborative approach involved:
- We coded one interview transcript and survey responses together to first align on the direction of the codes used
- We then each coded two interview transcripts separately
- We connected to review all codes together and iterated on them after the first pass of coding was completed
- We then connected a second time to review the codes again and iterated on them until we were in agreement with each code
Codebook
Coding survey responses and interview transcripts yielded:
Positionality
During the analysis process, in order to help keep preconceived notions and assumptions from influencing themes and conclusions from our data, we acknowledged our own backgrounds in finance and use of mobile investing applications as potential causes of bias and set aside the findings and patterns we anticipated to see in our data. Additionally, all codes, themes, interpretations, and key findings were discussed together as project partners to help limit individual biases from emerging. Themes and conclusions also needed to tie back to our research purpose and questions and the data that did not was excluded from final themes and key findings, which helped prevent us from forcing patterns and data to fit our research purpose and questions.
Key Findings
Key findings fell into five themes:
Discussion
What is working well?
Through conducting this research study, we learned that our participants think mobile investing applications work well overall and meet most of their investing needs. Despite some pain points, using mobile investing applications seems to be a net positive experience for users we interviewed.
We heard that users appreciate the convenience of being able to access their investment portfolios while on-the-go, see important information at a glance, and value the ease of use and user-friendly interfaces provided by mobile investment platforms. Users of mobile investment apps are able to make informed investment decisions without being experts in finance or professional investors, offering accessibility for less experienced investors.
We also heard that users of mobile investing apps are able to invest money for the future and work toward long-term and short-term financial goals. Being able to maintain different types of accounts and have access to a wide variety of investment options is also appreciated by users. And mobile investment app users value having access to community resources and seek out additional information from other users online on social media and discussion forums as part of their research process.
Supportive customer service agents also seem to increase satisfaction among mobile investment app users. We heard that users appreciate being able to discuss issues with a real human representative, as opposed to an AI assistive chatbot. Sufficiently addressing user issues while mitigating frustrations and concerns should be a continued practice in order to ensure continued user satisfaction.
What needs improvement?
Users think mobile investing apps work well and are user-friendly, but we heard they also think these applications have some room for improvement. In order to address the pain points experienced by users and enhance the mobile investment experience, we recommended the following four key improvements:
What else did we learn?
During this research study, we also heard that users will sometimes utilize multiple mobile investing applications to take advantage of the differences amongst platforms. Some examples of this practice that were mentioned during interviews included using a secondary platform for simpler investment charts or using a secondary platform to place trades that are restricted on a user’s primary application of use.
In addition to our four key recommended improvements, we urged mobile investment platforms to do research about their user base to explore what other secondary investment apps their users may be using to compensate for shortfalls experienced while using their primary mobile investment app.
With the data collected, mobile investment platforms can make additional changes based on their unique user groups.
Limitations and Next Steps
Limitations of our research included:
Given our limitations, we view our work as more of a preliminary study. We encourage future work to consider incorporating the following changes for stronger findings and more confidence for formal research studies: run this study with more participants consisting of a less convenient sample of users of mobile investing applications and adhere to a scope focused on a single mobile investing application or a predetermined set of investing applications to gain deeper understanding and insights.
Reflection
Looking back, what would i have done differently?
If I could go back, I would likely pursue a narrower scope for this research project. By pursuing a scope involving just one mobile investment application, rather than several, I anticipate that we would have had a better chance at possibly collecting more valuable data and gaining deeper insights and understandings from that data. While I still think we were able to address our research questions and meet our research purpose, the themes and findings we were able to develop didn’t feel quite as meaningful or novel as we had hoped alongside the existing research that has already been conducted about mobile investment applications. But with this being my first attempt at conducting a qualitative research project, the research study design and methodology we ended up pursuing over the course of the semester still offered a great learning experience!
What did I learn from working on this project?
Through working on this qualitative research project, I learned that the research process, particularly interviewing participants and coding transcripts, is such a craft and one that requires practice. But with this project under my belt, I feel more confident in my user research skills. Additionally, this project taught me that while I’ve decided to pursue a future in UX and design, there will be opportunities to leverage my background in economics and finance, as well as my interests in these areas.
How was my experience working on this study?
Working on this project with Ahona, my research partner, was a rewarding experience. My respect for her research skills and collaborative mindset has only increased since we began working on this project. Despite being in different areas and needing to meet virtually, we were determined to conduct a small-scale study we could be proud of and with our combined efforts, I think we did just that!
References
[1] Gary Shorter. 2020. Robinhood, the Fintech Discount Broker: Recent Developments and Concerns. Congressional Research Service, 1-3. [2] H. Kent Baker and Victor Ricciardi. 2014. Investor Behavior: The Psychology of Financial Planning and Investing. John Wiley & Sons. [3] James Pasztor. 2021. Robinhood - Hero or Villain?. Journal of Financial ServiceProfessionals, 75(5): 18-22. [4] Jason Fernando. 2022. What Is Financial Literacy, and Why Is It SoImportant?. Investopedia. [5] Lu Fan. 2022. Mobile Investment Technology Adoption Among Investors. The International Journal of Bank Marketing, 40(1): 50-67. [6] Michael Dickard. 2020. Cognitive Biases in Retail Investor Decision Making and HCI: A Research Agenda. The Vanguard Group. 1-7. [7] Ralph L. Rosnow. 1991. Inside rumor: A personal journey. American Psychologist, 46(5): 484–496.[8] Satish Kumar and Nisha Goyal. 2015. Behavioural biases in investment decision making – a systematic literature review. Qualitative Research in Financial Markets, 7(1): 88–108. [9] Sayan Chaudhry and Chinmay Kulkarni. 2021. Design Patterns of Investing Apps and Their Effects on Investing Behaviors. DIS’ 21: Designing Interactive Systems Conference 2021, 777–788.