Showing posts with label large. Show all posts
Showing posts with label large. Show all posts

Sunday, September 23, 2012

3rd Research in the large workshop, this time at MobileHCI2012

Having visited already the previous serious of Research in the Large (2010, 2011) I was very much looking forward to meeting the large researchers again investing approaches, tools, research questions for conducting research by deploying apps.
This year I was talking about three topics. In the first paper [1] we investigated the compliance of users of updating their mobile apps. We analyzed the lifecycle of various versions of a released app and found that after one week still half of the users run an old version. Thus, we recommend that developers actively notify their users inside the app to conduct an update.
Throughout the workshop we received interesting comments regarding whether could differentiate within the non-updaters between active and no-longer active users. Furthermore, it could be interesting to cluster the users by age, categories of apps etc. in order to find out whether updaters and non-updaters share certain characteristics. Finally, it would be interesting to track whether the newly introduced 'update all' feature in Android increases the compliance. We might also collect updates histories of apps from appbrain.com and compare our results with other apps.
Then I talked about our analysis of apps [2]  (based on appaware.com stats) available in the Android market. We analyzed the volumes of apps being downloaded corresponding to the overall sales. In contrast to amazon which largely live on the longtail of products, the Android market clearly is a hit market, where the most popular products contribute the largest parts of the sales. Our conclusion was that niche apps are still hard to find on the Android market (in contrast to the recommender systems revealing niches in amazon), thus research should focus on developing recommender systems. Furthermore, we found there is a significant amount apps generating large revenues with rather low download numbers by premium prices. Thus, developers should consider to increase the price of their apps significantly.
Finally, I presented NFC Heroes [3] we motivates users by a card trading fight game to upload information about deployed NFC tags.
The feedback we received from the workshop to give more hints to users where NFC tags are, e.g. information about local deployment activities as NFC tags at bus stops in San Francisco. Later, we might also building a service listing places and services of NFC tags in order to educate and inform users where they could use the NFC feature. Finally, we should consider to also listen to NFC intends  from other apps on the phone in order to capture any NFC interaction of the user's phone.

 In the workshop we had 7 talks in total which all latest for only 7min followed by 7min of discussion. This gave us enough time in the afternoon for some more in depth discussion about how to recruit users and what to learn form app stores.
To start off the discussion Frank Bentley first motivated why to do field studies at all (compared to lab studies): investigate real usage, impact of new systems, usability, inspiration, qualitative understanding, or quantitative confirmation of a previous small study.
Then Frank listed various option for recruiting users: in addition to the practice of small studies via mechanical turk, craigslist, posters in stores, snowball sampling, classmates and lab-mates, in the large we can populate the what's new section app stores, app store ads, targeted Facebook ads, re-tweets, personal tweets, presentation, and advertising. Another idea mentioned was to treat user's privacy as the user currency, the user owns and can decide to release for receiving specific services/benefits in return. Finally, our discussions revealed that either way user are recruited it is essential that papers clearly document how this was done. This is the only way that readers and reviewers can judge on the validity of the results.
The second part of the discussion focused on what kind of data and which research questions could be specifically tackled by doing research using app stores. On the one hand we get stats about downloads and locations of apps, logs of button presses, and partly demographics. However, what we don't get is the context of use and insights into why people did or didn't an activity. Overall, we are limited to tasks users in the wild want to do which may limit to tasks which are fun or particularly useful. Finally, a remaining question is as the number of apps in the markets develops and matures, whether the effort of releasing a decent app would be still feasible in near future.
David Aymann Shamma reported about his experience collecting and analyzing data from a chatting app for watching videos remotely together. David focused a lot on responsibility and ethics of treating the data being gathered throughout the project. Handling this massive amounts of data and also highly sensitive data would be a new dimension of what we as researchers would be usually used to. David also recommended to be generous and consider deleting all the session which appear to be odd or incomplete.

Overall, I really enjoyed meeting all these people sharing an interest for doing research in the large. I'm looking forward to help running this next year again. It's probably also time to come with a manifesto or guidelines describing how to do research in the large. Collective experience of researchers in the domain (e.g. [4]) should motivate the community to consider this new research method going far beyond the established modes of user studies.

[1] Update Behavior in App Markets and Security Implications: A Case Study in Google Play, Andreas Möller, Florian Michahelles, Stefan Diewald, Luis Roalter, Matthias Kranz, 3rd workshop on Research in the large (at MobileHCI 2012), San Francisco, September 21, 2012. [pdf]
[2] Examining the Long Tail of Android Market, Nan Zhong, Florian Michahelles, 3rd workshop on Research in the large (at MobileHCI 2012), San Francisco, September 21, 2012. [pdf]
[3] NFC Heroes – Observing NFC Adoption through a Mobile Trading Card Game, Lukas Murmann, Florian Michahelles, Matthias Kranz, 3rd workshop on Research in the large (at MobileHCI 2012), San Francisco, September 21, 2012. [pdf]
[4] My App is My Experiment: Experience Studiesin Mobile App Stores, Henze, N., Pielot, M., Poppinga, B., Schinke, T., & Boll, S. (2011). International Journal of Mobile Human Computer Interaction , 3 (4), 21.

Monday, September 27, 2010

Ubicomp in the large


Finally, there was a workshop organized by MobileLifeCentre which I considered really important: how to conduct ubicomp in the large considering all these app stores and mobile devices...

First the participants had 3-5 minutes to briefly report about their work and experience with ubicomp in the large. Examples ranged from social soccer fan apps [1] to architecture planning [2] and public displays [3]. Myself I briefly introduced Appaware but quickly realized that most of the people already knew about it when they showed it to me on their phones.

During the second part of the workshop we had a lively discussion about
  • how to get users?
  • how to keep control of the quality of the results?
  • which research questions suitable to be investigated in the large?
Most importantly, the apps to be distributed have to create some value for the users, be it a game or a productivity app. Hitting the blogosphere is great which is probably best achieved by a well-designed app.
Keeping control of the data quality probably is a tricky one as we do not know who the users actually are: a bunch of people with nicknames. This might overcome by linking facebook profiles to the app, but this requires some added-value functionality of the app justifying this link. Throughout the discussions we found out that with every question we add to an embedded questionaire we limit and design the target group of users: only a subsets likes to answer - pure self-selection. Giving rewards and payment might be an incentive, but this again also incentivizes another special subset of users. There is probably nothing to be done than to try to capture as much information as possible from the users which has to be done in a transparent way with opt-out.
Finally, we struggled about why doing research in the large at all?
Reason were: because it matters more, gets closer to reality, finds unexpected uses, promotes the own lab, creates longer lasting results, gives something back to the public, yields more justified statistics...
In order to distinguish from pure app developments there should be a research question beyond simply finding out whether the crowd would accept a certain app.

I'm looking forward to the special issue following up on this topics soon to appear in IJMHCI.
Thanks to Henriette Cramer, Nicolas Belloni, Mattias Rost, Frank Bentley and Didier Chincholle for this great workshop!