Market Analysis Cheat Sheet

Session 1 - Segementation

1. Data Types:
   1. Geo-demographics - 用户基础信息
   2. Psychograhics - 用户心理
   3. Behavorial - 用户行为
   4. Benefits & Needs - 用户需求
   5. 
2. How (to do segementation) ? - Cluster analysis(聚类)
   1. Hierarchical Clustering - Recursively group entities based on how similar they are
      1. 计算所有点之间的举例 (Euclidean Distance)
      2. Select Min {Dij} and join i and j at that distance
      3. 如何计算剩余的点到已经构成的组的距离？
         1. Minimum (single) linkage - Distance to Closest Point
         2. Average linkage - Average Distance over All Points
         3. Maximum (complete) linkage - Distance to Furthest point
         4. Ward linkage - Minimize the within-cluster variance
            1. Add {4} to {1,2} to form cluster {1,2,4} ， Distance = variance of {1,2,4} – (variance of {1,2} + variance of {4})
   2. K-Means - Minimize within-cluster variance, maximize between cluster variance (k centers)
      1. Initialize centroids(centers) (End Result depends on initialization)
      2. Assign points (observations) to the nearest centroid
      3. Re-compute centers
      4. Stop when no change
      5. K = ? though ?
         1. Inertia - measure of how internally coherent clusters are, lower = better (Always decreases with the number of clusters)
         2. 
         3. Elbow Plot: Increase the number of clusters and monitor the inertia
         4. Ratio Plot: Increase the number of clusters and monitor (total between sum of squares/total sum of squares)
         5. 这些统称为Determined by Fit, 另一种方法是Determine by Interpretability (capture meaningful differences)
         6. Characteristics of ideal segments: Large, Identifiable, Distinctive, Stable(LIDS), more importantly - actionable
      6. Chi-square Test
         1. Determine whether a difference between two categorical variables is due to chance or a relationship between them
         2. Expected count = (row total) * (column total) / total sample size
         3. 
         4. with degrees of freedom = (# of rows - 1)(# of columns - 1)
         5. Reject the null when p-value of

Session 6

Utility function

1. Consumers preferences for alternatives are represented by utility functions. Rational consumers choose the alternative with the highest utility.
2. Utility = F(Consumer Characteristics，Alternative Attributes) is deterministic(consistent)
3. To simulate real world inconsistency, we add $$ {U}{ij} = {V}{ij} +{\epsilon}_{ij} $$ where e_ij represents total impact of all unobserved attributes and demographics relevant to a given choice occasion(stochastic part)
4. For ‘Alternative Attributes’ - coefficients are the same across alternatives
5. For ‘Character Charateristic’ - coefficients are the different across alternatives
6. e_ij varies across alternatives j and across consumers i, but can be assumed coming from a probability distribution (Gumbel distribution)
7. 所以Customer i 选择选项 j 的概率是 exp(i 对 j的utility) / exp(i 对 所有选项的utility之和)
8. Identification: Only differences in utility matter
   1. Need to set one alternative specific constant to zero
   2. Need to set the coefficients of the individual characteristics to zero for one alternative

Elastics & IIA

Session 7

4 Basic Approach: Simple Summaries, Sentiment Analysis, Topic Modeling, Large Language Models.

Sentiment Analysis

Lexicon-based Sentiment Analysis = Classification of words

1. One popular choice LIWC = Linguistic Inquiry and Word Count
2. Logistic Regression on Y = churn and X = LIWC Proportions (Page 122)

Topic Modeling

Automatic summarization of documents through topics(set of commonly co-occurring words)
Most Common Model = Latent Dirichlet Allocation (LDA)

1. Output 1: Which words belong to which topics (i.e., what are the topics)?
2. Output 2: Which topics best describe each documenti.e., what percentage of the words in a given document are from topic 1, topic 2, …)?
3. Perplexity = measure of predictive performance for a language model (Lower = Better)
   

Large Language Model

A category of foundation models (large deep learning model trained on generalized and unlabeled data and used as a starting point for other models) trained on immense amounts of data making them capable of understanding and generating natural language and other types of content to perform a wide range of tasks

1. Word Embeddings = representation of a word as a vector of numbers (Able to perform word algebra)
   1. Similar meaning = similar representation
2. Some uses in marketing
   1. Text summarization – summarizing customer reviews, complaints, etc.
   2. Sentiment analysis – extracting more nuanced sentiment (e.g., granular emotions)
   3. Text generation – creating product descriptions, social media posts, emails

Session 8

products are bundles of independent attributes “Products = the sum of their parts”
Attributes is consist of levels. Combination of levels form profile. Value derived from a level is part-worth. Total part-worth is utility.

1. How do we estimate consumers’ part-worths?
   1. Step 1: Ask them to rate many potential profiles.
      1. Which profiles? Fractional factorial design - The minimal number of questions to get the information we need(Design determined by the number of attributes / levels)
   2. Step 2: Analyze the data!

Conjoint analysis

1. Rating based Conjoint = Multiple Regression - Result Coefficients = Part-worth
   1. Baseline = 0
   2. Importance = Range (Range / Sum of Range)
   3. We can use these coefficients to predit newcomers
2. Limitations: You can never include all the attributes
3. Other Approaches
   1. Eye Tracking
   2. Learn Preferences Faster / Better

Diffusion of Innovation

1. The Bass model - Predict adoption curve, Number of adopters in period " = Adoption rate in period " x Number of potential adopters in period "(传染模型)
2. Application: Can predict both our innovation and a innovation we relies on
   
   

Session 9

Bass model

m = Demographic data， p, q: Historical analysis of analogous innovations
Factors to take into account when evaluating analogies: • Environmental situation • Market structure • Buyer behavior • Marketing mix strategy • Characteristics of innovation itself
Diffusion Speed Has Generally Increased Over Time

Generative AI

生成式人工智能（Generative AI） 是一种能够生成新内容（如文本、图像、音频、视频等）的人工智能技术。与传统的判别式AI不同，生成式AI不仅能够识别和分类数据，还能基于已有的数据创作出全新的、原创性的内容

1. Generative Models
   1. Generative Adversarial Networks (GANs) 生成对抗网络
      GANs由生成器（Generator）和判别器（Discriminator）组成，生成器负责生成逼真的数据样本，判别器则区分生成的数据与真实数据。(图像相关)
   2. Variational Autoencoders (VAEs) 变分自编码器
      通过编码器将输入数据映射到潜在空间，再通过解码器从潜在空间生成新数据 (数据相关)
   3. Transformers 基于注意力机制的深度学习模型
      Transformer 是一种用于处理序列数据（如文本、音频、时间序列等）的神经网络架构。Transformer 完全基于注意力机制（Attention Mechanism），无需依赖序列的顺序处理，从而实现更高的并行化效率和更好的性能。
2. Applications in marketing
   1. Perceptual Maps (Classification)
   2. WTP
      1. Prompt Engineering - providing clear instructions to a generative model to get what you want

Explainable AI

Session 10

Pricing + Placing

Session 11

A/B test

Session 12

Bias & Fairness

源课件下载

Market Analysis Session 1
Market Analysis Session 2-5
Market Analysis Session 6
Market Analysis Session 6-12
Business Analysis Session 1-10