publications

2024

1. IEEE IoT J

   Kiosk Recommend System Based On Self-Supervised Representation Learning of User Behaviors in Offline Retail

   Nam Gyu Jung , Van Thuy Hoang, O-Joun Lee, and Chang Choi

   IEEE Internet of Things Journal, 2024

   Abs

   Recently, in the offline distribution field, as the number of data collection and analysis cases increases by applying IoT devices to kiosks, research on hyper-personalized recommendation systems has become critical. Recommendation systems only work well in some data-rich areas (industries). Therefore, it is unsuitable for kiosk systems with multiple domains and data imbalances, and it is challenging to collect detailed information such as user reviews and product descriptions. In this paper, we propose a context-aware hyper-personalized recommendation system that utilizes context information collected from kiosk IoT devices, minimizes the model size of the kiosk device, and aims for consistent performance and high recommendation performance in various domains. We also developed effective self-supervised learning to increase data learning efficiency in data imbalance environments. The quality of products recommended by the proposed kiosk recommendation system was evaluated using transactions that occurred in an actual kiosk system. As a result, compared to the existing recommendation system, all performance indicators improved by an average of 20%. When the self-supervised learning method was additionally applied, it improved by an average of 0.8% more. In particular, it shows superior performance regarding the quality of recommended items and resource usage according to users. IEEE

2. IEEE Sens J

   Quantification of dysnatremia using single-beam acoustic microbeam and convolutional neural networks

   Ji Won Nam , Hyeon-Ju Jeon, Jeong Eun Lee , O-Joun Lee, and Hae Gyun Lim

   IEEE Sensors Journal, 2024

   Abs

   Recently, the use of artificial intelligence (AI) in cell analysis has gained significant attention, with particular focus on ultrasound-based AI for single-cell analysis. One application is diagnosing diseases by using ultrasound signals to analyze the physical properties contained in the signals. Dysnatremia, which can result in severe consequences to health such as stroke and cardiovascular disease, can be measured using blood sodium tests. However, these tests are performed by drawing blood, and obtaining the results requires a considerable amount of time. In addition, it has low reliability because the results vary depending on the inspection equipment and inspection method. In this study, we propose a novel approach for the quantification of dysnatremia using a single-beam acoustic microbeam (SBAM) and convolutional neural networks (CNNs). A 90 MHz transducer was fabricated and used to obtain reflected signals from red blood cells (RBCs), which are affected by the shape of the cells which, in turn, depend on the sodium concentration. Blood samples with varying sodium chloride concentrations were tested, and the reflected signals were analyzed using a CNN, for automation as opposed to manual analysis. The accuracy of the classification of the blood samples into 10 and 5-level groups, based on the sodium chloride concentration, was 0.961 and 0.942, respectively, as determined using CNNs. The results of this study demonstrate the potential of SBAM and CNN technologies for efficient quantification of sodium concentration in blood. This technology will help to diagnose dysnatremia in a non-invasive way with reduced analysis time and high accuracy. IEEE

3. Ultrasonics

   Diagnosis of diabetes mellitus using high frequency ultrasound and convolutional neural network

   Jeong Eun Lee , Hyeon-Ju Jeon, O-Joun Lee, and Hae Gyun Lim

   Ultrasonics, 2024

   Abs

   The incidence of diabetes mellitus has been increasing, prompting the search for non-invasive diagnostic methods. Although current methods exist, these have certain limitations, such as low reliability and accuracy, difficulty in individual patient adjustment, and discomfort during use. This paper presents a novel approach for diagnosing diabetes using high-frequency ultrasound (HFU) and a convolutional neural network (CNN). This method is based on the observation that glucose in red blood cells (RBCs) forms glycated hemoglobin (HbA1c) and accumulates on its surface. The study incubated RBCs with different glucose concentrations, collected acoustic reflection signals from them using a custom-designed 90-MHz transducer, and analyzed the signals using a CNN. The CNN was applied to the frequency spectra and spectrograms of the signal to identify correlations between changes in RBC properties owing to glucose concentration and signal features. The results confirmed the efficacy of the CNN-based approach with a classification accuracy of 0.98. This non-invasive diagnostic technology using HFU and CNN holds promise for in vivo diagnosis without the need for blood collection. © 2023

4. BMC Med Img

   Automated classification of liver fibrosis stages using ultrasound imaging

   Hyun-Cheol Park , YunSang Joo , O-Joun Lee, Kunkyu Lee , Tai-Kyong Song , Chang Choi , Moon Hyung Choi , and Changhan Yoon

   BMC Medical Imaging, 2024

   Abs

   Background: Ultrasound imaging is the most frequently performed for the patients with chronic hepatitis or liver cirrhosis. However, ultrasound imaging is highly operator dependent and interpretation of ultrasound images is subjective, thus well-trained radiologist is required for evaluation. Automated classification of liver fibrosis could alleviate the shortage of skilled radiologist especially in low-to-middle income countries. The purposed of this study is to evaluate deep convolutional neural networks (DCNNs) for classifying the degree of liver fibrosis according to the METAVIR score using US images. Methods: We used ultrasound (US) images from two tertiary university hospitals. A total of 7920 US images from 933 patients were used for training/validation of DCNNs. All patient were underwent liver biopsy or hepatectomy, and liver fibrosis was categorized based on pathology results using the METAVIR score. Five well-established DCNNs (VGGNet, ResNet, DenseNet, EfficientNet and ViT) was implemented to predict the METAVIR score. The performance of DCNNs for five-level (F0/F1/F2/F3/F4) classification was evaluated through area under the receiver operating characteristic curve (AUC) with 95% confidential interval, accuracy, sensitivity, specificity, positive and negative likelihood ratio. Results: Similar mean AUC values were achieved for five models; VGGNet (0.96), ResNet (0.96), DenseNet (0.95), EfficientNet (0.96), and ViT (0.95). The same mean accuracy (0.94) and specificity values (0.96) were yielded for all models. In terms of sensitivity, EffcientNet achieved highest mean value (0.85) while the other models produced slightly lower values range from 0.82 to 0.84. Conclusion: In this study, we demonstrated that DCNNs can classify the staging of liver fibrosis according to METAVIR score with high performance using conventional B-mode images. Among them, EfficientNET that have fewer parameters and computation cost produced highest performance. From the results, we believe that DCNNs based classification of liver fibrosis may allow fast and accurate diagnosis of liver fibrosis without needs of additional equipment for add-on test and may be powerful tool for supporting radiologists in clinical practice. © The Author(s) 2024.

2023

1. EACL

   “Why do I feel offended?’ Korean Dataset for Offensive Language Identification

   San-Hee Park , Kang-Min Kim , O-Joun Lee, Youjin Kang , Jaewon Lee , Su-Min Lee , and SangKeun Lee

   2023

   Abs

   Offensive content is an unavoidable issue on social media. Most existing offensive language identification methods rely on the compilation of labeled datasets. However, existing methods rarely consider low-resource languages that have relatively less data available for training (e.g., Korean). To address these issues, we construct a novel KOrean Dataset for Offensive Language Identification (KODOLI). KODOLI comprises more fine-grained offensiveness categories (i.e., not offensive, likely offensive, and offensive) than existing ones. A likely offensive language refers to texts with implicit offensiveness or abusive language without offensive intentions. In addition, we propose two auxiliary tasks to help identify offensive languages: abusive language detection and sentiment analysis. We provide experimental results for baselines on KODOLI and observe that pre-trained language models suffer from identifying "LIKELY" offensive statements. Quantitative results and qualitative analysis demonstrate that jointly learning offensive language, abusive language and sentiment information improves the performance of offensive language identification. © 2023 Association for Computational Linguistics.

2. Sensors

   Graph Representation Learning and Its Applications: A Survey

   Van Thuy Hoang, Hyeon-Ju Jeon, Eun-Soon You , Yoewon Yoon , Sungyeop Jung , and O-Joun Lee

   Sensors, 2023

   Abs

   Graphs are data structures that effectively represent relational data in the real world. Graph representation learning is a significant task since it could facilitate various downstream tasks, such as node classification, link prediction, etc. Graph representation learning aims to map graph entities to low-dimensional vectors while preserving graph structure and entity relationships. Over the decades, many models have been proposed for graph representation learning. This paper aims to show a comprehensive picture of graph representation learning models, including traditional and state-of-the-art models on various graphs in different geometric spaces. First, we begin with five types of graph embedding models: graph kernels, matrix factorization models, shallow models, deep-learning models, and non-Euclidean models. In addition, we also discuss graph transformer models and Gaussian embedding models. Second, we present practical applications of graph embedding models, from constructing graphs for specific domains to applying models to solve tasks. Finally, we discuss challenges for existing models and future research directions in detail. As a result, this paper provides a structured overview of the diversity of graph embedding models. © 2023 by the authors.

3. IEEE Access

   Companion Animal Disease Diagnostics Based on Literal-Aware Medical Knowledge Graph Representation Learning

   Van Thuy Hoang, Thanh Sang Nguyen , Sangmyeong Lee , Jooho Lee , Luong Vuong Nguyen , and O-Joun Lee

   IEEE Access, 2023

   Abs

   Knowledge graph (KG) embedding has been used to benefit the diagnosis of animal diseases by analyzing electronic medical records (EMRs), such as notes and veterinary records. However, learning representations to capture entities and relations with literal information in KGs is challenging as the KGs show heterogeneous properties and various types of literal information. Meanwhile, the existing methods mostly aim to preserve graph structures surrounding target nodes without considering different types of literals, which could also carry significant information. In this paper, we propose a knowledge graph embedding model for the efficient diagnosis of animal diseases, which could learn various types of literal information and graph structure and fuse them into unified representations, namely LiteralKG. Specifically, we construct a knowledge graph that is built from EMRs along with literal information collected from various animal hospitals. We then fuse different types of entities and node feature information into unified vector representations through gate networks. Finally, we propose a self-supervised learning task to learn graph structure in pretext tasks and then towards various downstream tasks. Experimental results on link prediction tasks demonstrate that our model outperforms the baselines that consist of state-of-the-art models. © 2023 The Authors.

4. RACS

   Long Short-Term Memory Network (LSTM) based Stock Price Prediction

   Akshat Gaurav , Varsha Arya , Kwok Tai Chui , Brij B. Gupta , Chang Choi , and O-Joun Lee

   2023

   Abs

   Predicting stock prices is a challenging and highly sought-after task in financial markets. In recent years, deep learning techniques, particularly Long Short-Term Memory (LSTM) networks, have shown promising results in capturing complex temporal dependencies and forecasting time series data. This research paper presents a LSTM-based framework for stock price prediction. The proposed framework utilizes historical stock price data. The LSTM model is designed to learn the underlying patterns and trends in the data, enabling it to make accurate predictions of future stock prices. We preprocess the data, including normalization and feature engineering, to enhance the model’s ability to extract meaningful patterns. We employ appropriate evaluation metrics, such as mean squared error (MSE) and root mean squared error (RMSE), to assess the accuracy of the predictions. Experimental results demonstrate that the LSTM-based framework achieves competitive performance in stock price prediction compared to traditional statistical models and other machine learning approaches. © 2023 ACM.

5. IEEE Access

   Classification of Liver Fibrosis From Heterogeneous Ultrasound Image

   Yunsang Joo , Hyun-Cheol Park , O-Joun Lee, Changhan Yoon , Moon Hyung Choi , and Chang Choi

   IEEE Access, 2023

   Abs

   With the advances in deep learning, including Convolutional Neural Networks (CNN), automated diagnosis technology using medical images has received considerable attention in medical science. In particular, in the field of ultrasound imaging, CNN trains the features of organs through an amount of image data, so that an expert-level automatic diagnosis is possible only with images of actual patients. However, CNN models are also trained on the features that reflect the inherent bias of the imaging machine used for image acquisition. In other words, when the domain of data used for training is different from that of data applied for an actual diagnosis, it is unclear whether consistent performance can be provided by the domain bias. Therefore, we investigate the effect of domain bias on the model with liver ultrasound imaging data obtained from multiple domains. We have constructed a dataset considering the manufacturer and the year of manufacturing of 8 ultrasound imaging machines. First, training and testing were performed by dividing the entire data, in a commonly used method. Second, we have utilized the training data constructed according to the number of domains for the machine learning process. Then we have measured and compared the performance on internal and external domain data. Through the above experiment, we have analyzed the effect of domains of data on model performance. We show that the performance scores evaluated with the internal domain data and the external domain data do not match. We especially show that the performance measured in the evaluation data including the internal domain was much higher than the performance measured in the evaluation data consisting of the external domain. We also show that 3-level classification performance is slightly improved over 5-level classification by mitigating class imbalance by integrating similar classes. The results highlight the need to develop a new methodology for mitigating the machine bias problem so that the model can work correctly even on external domain data, as opposed to the usual approach of constructing evaluation data in the same domain as the training data. © 2013 IEEE.

2022

1. SciRep

   Automated estimation of cancer cell deformability with machine learning and acoustic trapping

   O-Joun Lee, Hae Gyun Lim, K. Kirk Shung , Jin-Taek Kim , and Hyung Ham Kim

   Scientific Reports, 2022

   Abs

   Cell deformability is a useful feature for diagnosing various diseases (e.g., the invasiveness of cancer cells). Existing methods commonly inflict pressure on cells and observe changes in cell areas, diameters, or thickness according to the degree of pressure. Then, the Young’s moduli (i.e., a measure of deformability) of cells are estimated based on the assumption that the degrees of the changes are inversely proportional to Young’s moduli. However, manual measurements of the physical changes in cells are labor-intensive, and the subjectivity of the operators can intervene during this step, thereby causing considerable uncertainty. Further, because the shapes of cells are nonuniform, we cannot ensure the assumption for linear correlations of physical changes in cells with their deformability. Therefore, this study aims at measuring non-linear elastic moduli of live cells (degrees of cell deformability) automatically by employing conventional neural networks (CNN) and multilayer perceptrons (MLP) while preserving (or enhancing) the accuracy of the manual methods. First, we obtain photomicrographs of cells on multiple pressure levels using single-beam acoustic tweezers, and then, we suggest an image preprocessing method for emphasizing changes in cell areas on the photomicrographs. The CNN model is trained to measure the ratios of the cell area change at each pressure level. Then, we apply the multilayer perceptron (MLP) to learn the correlations of the cell area change ratios according to the pressure levels with cell deformability. The accuracy of the CNN was evaluated using two types of breast cancer cells: MDA-MB-231 (invasive) and MCF-7 (noninvasive). The MLP was assessed using five different beads (Young’s moduli from 0.214 to 9.235 kPa), which provides standardized reference data of the non-linear elastic moduli of live cells. Finally, we validated the practicality of the proposed system by examining whether the non-linear elastic moduli estimated by the proposed system can distinguish invasive breast cancer cells from noninvasive ones. © 2022, The Author(s).

2. Sensors

   Day-Ahead Hourly Solar Irradiance Forecasting Based on Multi-Attributed Spatio-Temporal Graph Convolutional Network

   Hyeon-Ju Jeon, Min-Woo Choi , and O-Joun Lee

   Sensors, 2022

   Abs

   Solar irradiance forecasting is fundamental and essential for commercializing solar energy generation by overcoming output variability. Accurate forecasting depends on historical solar irradiance data, correlations between various meteorological variables (e.g., wind speed, humidity, and cloudiness), and influences between the weather contexts of spatially adjacent regions. However, existing studies have been limited to spatiotemporal analysis of a few variables, which have clear correlations with solar irradiance (e.g., sunshine duration), and do not attempt to establish atmospheric contextual information from a variety of meteorological variables. Therefore, this study proposes a novel solar irradiance forecasting model that represents atmospheric parameters observed from multiple stations as an attributed dynamic network and analyzes temporal changes in the network by extending existing spatio-temporal graph convolutional network (ST-GCN) models. By comparing the proposed model with existing models, we also investigated the contributions of (i) the spatial adjacency of the stations, (ii) temporal changes in the meteorological variables, and (iii) the variety of variables to the forecasting performance. We evaluated the performance of the proposed and existing models by predicting the hourly solar irradiance at observation stations in the Korean Peninsula. The experimental results showed that the three features are synergistic and have correlations that are difficult to establish using single-aspect analysis. © 2022 by the authors.

3. SciRep

   Automated cell-type classification combining dilated convolutional neural networks with label-free acoustic sensing

   Hyeon-Ju Jeon, Hae Gyun Lim, K. Kirk Shung , O-Joun Lee, and Min Gon Kim

   Scientific Reports, 2022

   Abs

   This study aimed to automatically classify live cells based on their cell type by analyzing the patterns of backscattered signals of cells with minimal effect on normal cell physiology and activity. Our previous studies have demonstrated that label-free acoustic sensing using high-frequency ultrasound at a high pulse repetition frequency (PRF) can capture and analyze a single object from a heterogeneous sample. However, eliminating possible errors in the manual setting and time-consuming processes when postprocessing integrated backscattering (IB) coefficients of backscattered signals is crucial. In this study, an automated cell-type classification system that combines a label-free acoustic sensing technique with deep learning-empowered artificial intelligence models is proposed. We applied an one-dimensional (1D) convolutional autoencoder to denoise the signals and conducted data augmentation based on Gaussian noise injection to enhance the robustness of the proposed classification system to noise. Subsequently, denoised backscattered signals were classified into specific cell types using convolutional neural network (CNN) models for three types of signal data representations, including 1D CNN models for waveform and frequency spectrum analysis and two-dimensional (2D) CNN models for spectrogram analysis. We evaluated the proposed system by classifying two types of cells (e.g., RBC and PNT1A) and two types of polystyrene microspheres by analyzing their backscattered signal patterns. We attempted to discover cell physical properties reflected on backscattered signals by controlling experimental variables, such as diameter and structure material. We further evaluated the effectiveness of the neural network models and efficacy of data representations by comparing their accuracy with that of baseline methods. Therefore, the proposed system can be used to classify reliably and precisely several cell types with different intrinsic physical properties for personalized cancer medicine development. © 2022, The Author(s).

2021

1. Comp4Human

   Computing4Human 2021: The 2nd international conference on human-centered artificial intelligence

   Nam D. Vo , O-Joun Lee, Khac-Hoai Nam Bui , Hae Gyun Lim, Hyeon-Ju Jeon, Phuong-Mai Nguyen , Jin-Taek Kim , Bui Quang Tuyen , Jason J. Jung, and Thuy Anh Vo

   2021

   Abs

   The 2nd international conference on human-centered artificial intelligence (Computing4Human) is hosted by the University of Economics, The University of Danang, on the 28th of October 2021. This conference tackled various research areas: the Computational Humanities and Social Sciences, Smart Infrastructure, Smart Healthcare, Secure and Green IoT Communications, Explainable Recommendation and Retrieval, and Business Intelligence. We collected 35 papers, which were fully refereed and underwent a single-blind review process by at least three reviewers. Finally, 26 papers have been published for six technical tracks. © 2021 CEUR-WS. All rights reserved.

2. MIS

   Interinstitutional Research Team Formation Based on Bibliographic Network Embedding

   O-Joun Lee, Seungha Hong , and Jin-Taek Kim

   Mobile Information Systems, 2021

   Abs

   This study aims at forming research teams for interinstitutional collaborations. Research institutes have their own purposes and topics of interest. Thus, supporting joint research between multiple institutes, we have to consider not only synergies between scholars but also purposes of the institutes. To solve this problem, we propose a bibliographic network embedding method that can learn characteristics of institutes, not only of each scholar. First, we compose a bibliographic network that consists of scholars, publications, venues, research projects, and institutes. Collaboration styles and research topics of institutes and scholars are extracted by mining subgraphs from the bibliographic network. Then, vector representations of network nodes are learned based on occurrences of subgraphs on the nodes and neighborhoods of the nodes. Based on the vector representations, we train multilayer perceptrons (MLP) to assess collaboration probability between scholars affiliated in different institutes. For training the MLP, we suggest three strategies: (i) considering every collaboration, (ii) focusing on interinstitutional collaborations, and (iii) focusing on collaboration outcomes. To evaluate the proposed methods, we have analyzed research collaborations of POSTECH (Pohang University of Science and Technology) and RIST (Research Institute of Industrial Science and Technology) from 2011 to 2020. Then, we conducted the research team formation for joint research of the two institutes according to two purposes: pure research and commercialization research. © 2021 O-Joun Lee et al.

3. JOI

   Learning multi-resolution representations of research patterns in bibliographic networks

   O-Joun Lee, Hyeon-Ju Jeon, and Jason J. Jung

   Journal of Informetrics, 2021

   Abs

   This study aims at representing research patterns of bibliographic entities (e.g., scholars, papers, and venues) with a fixed-length vector. Bibliographic network structures rooted in the entities are incredibly diverse, and this diversity increases in the outstanding entities. Thus, despite their significant volume, the outstanding entities obtain minimal learning opportunities, whereas low-performance entities are over-represented. This study solves the problem by representing the patterns of the entities rather than depicting individual entities in a precise manner. First, we describe structures rooted in the entities using the Weisfeiler-Lehman (WL) relabeling process. Each subgraph generated by the relabeling process provides information on the scholars, kinds of papers they published, standards of venues in which the papers were published, and types of their collaborators. We assume that a subgraph depicts the research patterns of bibliographic entities, such as the preference of a scholar in choosing either a few highly impactful papers or numerous papers of moderate impact. Then, we simplify the subgraphs according to multiple levels of detailedness. Original subgraphs represent the individuality of the entities, and simplified subgraphs represent the entities sharing the same research patterns. In addition, simplified subgraphs balance the learning opportunities of high- and low-performance entities by co-occurring with both types of entities. We embed the subgraphs using the Skip-Gram method. If the results of the embedding represent the research patterns of the entities, the obtained vectors should be able to represent various aspects of the research performance in both the short-term and long-term durations regardless of the performances of the entities. Therefore, we conducted experiments for predicting 23 performance indicators during four time periods for four performance groups (top 1%, 5%, 10%, and all entities) using only the vector representations. The proposed model outperformed the existing network embedding methods in terms of both accuracy and variance. © 2020 Elsevier Ltd.

4. AppSci

   Plot structure decomposition in narrative multimedia by analyzing personalities of fictional characters

   O-Joun Lee, Eun-Soon You , and Jin-Taek Kim

   Applied Sciences (Switzerland), 2021

   Abs

   This study aims to decompose plot structures of stories in narrative multimedia (i.e., creative works that contain stories and are distributed through multimedia). Since a story is interwoven with main plots and subplots (i.e., primary and ancillary story lines), decomposing a story into multiple story lines enables us to analyze how events in the story are allocated and logically connected. For the decomposition, the existing studies employed character networks (i.e., social networks of characters that appeared in a story) and assumed that characters’ social relationships are consistent in a story line. However, these studies overlooked that social relationships significantly change around major events. To solve this problem, we attempt to use the changes for distinguishing story lines rather than suffer from the changes. We concentrate on the changes in characters’ social relationships being the result of changes in their personalities. Moreover, these changes gradually proceed within a story line. Therefore, we first propose features for measuring changes in personalities of characters: (i) Degrees of characters in character networks, (ii) lengths of dialogues spoken by characters, and (iii) ratios of out-degrees for in-degrees of characters in character networks. We supposed these features reflect importance, inner/outer conflicts, and activeness of characters, respectively. Since characters’ personalities gradually change in a story line, we can suppose that the features also show gradual story developments in a story line. Therefore, we conduct regression for each feature to discover dominant tendencies of the features. By filtering scenes that do not follow the tendencies, we extract a story line that exhibits the most dominant personality changes. We can decompose stories into multiple story lines by iterating the regression and filtering. Besides, personalities of characters change more significantly in major story lines. Based on this assumption, we also propose methods for discriminating main plots. Finally, we evaluated the accuracy of the proposed methods by applying them to the movies, which is one of the most popular narrative multimedia. © 2021 by the authors. Licensee MDPI, Basel, Switzerland.

5. Comp4Human

   Learning contextual representations of citations via graph transformer

   Hyeon-Ju Jeon, Gyu-Sik Choi , Se-Young Cho , Hanbin Lee , Hee Yeon Ko , Jason J. Jung, O-Joun Lee, and Myeong-Yeon Yi

   2021

   Abs

   This study aims at representing the citation based on the citation context extracted from the citation network. Researchers cite papers for various purposes to describe their arguments in a logical structure. Thus, citations have different roles depending on what structure they are cited in the paper. In this paper, we first present a definition of the citation context and initialize the embedding vector based on the citation order and location. Then, based on the graph transformer model, we learn contextual citation embeddings. To represent citation context, we consider the following three parts: (i) textual features of paper, (ii) positional features of the citation context, and (iii) structural features of the citation network by applying the self-attention mechanism. © 2021 CEUR-WS. All rights reserved.

6. Comp4Human

   Quantifying conflicts in narrative multimedia by analyzing visual storytelling techniques

   O-Joun Lee, Jin-Taek Kim , and Eun-Soon You

   2021

   Abs

   This study aims at measuring conflict degrees of each shot in visual narrative multimedia (e.g., movies and TV series) by analyzing visual storytelling techniques, such as camerawork. To describe incidents in stories, directors use the techniques as like as visual language. Thus, visual storytelling techniques used in a shot should be correlated with incidents shown by the shot. In this study, we first present various taxonomies of the visual storytelling techniques and discuss which techniques have more correlations with conflicts than the others. Then, based on usages of the techniques in each shot, we measure intensity of conflicts described by the shot. Finally, we validated correlations of visual storytelling techniques with stories’ content by examining correlations of the proposed conflict measurement with conflict degrees annotated by scholars and practitioners in film studies. Copyright © by the paper’s authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

2020

1. IJCAI

   Story embedding: Learning distributed representations of stories based on character networks

   O-Joun Lee, and Jason J. Jung

   2020

   Abs

   This study aims to represent stories in narrative works (i.e., creative works that contain stories) with a fixed-length vector. We apply subgraph-based graph embedding models to dynamic social networks of characters that appeared in stories (character networks). We suppose that interactions between characters reflect the content of stories. We discretize the interactions by discovering the subgraphs and learn representations of stories by predicting occurrences of the subgraphs in corresponding character networks. We find subgraphs rooted in each character on each scene in multiple scales, using the WL (Weisfeiler-Lehman) relabeling process. To predict occurrences of subgraphs, we apply two approaches: (i) considering changes in subgraphs according to scenes and (ii) focusing on subgraphs on the last scene. We evaluated the proposed models by measuring the similarity between real movies with vector representations that were generated by the models. © 2020 Inst. Sci. inf., Univ. Defence in Belgrade. All rights reserved.

2. Comp4Human

   Discovering mise-en-scène in movies by analyzing scripts

   O-Joun Lee, and Jin-Taek Kim

   2020

   Abs

   This study aims to analyze mise-en-scène in movies. Although the mise-en-scène includes all the entities that appear on the scene, we focus on metaphorical meanings of the entities. When entities are expressions of the same metaphorical symbol, lexical meanings of the entities will be relevant to each other. Also, if a metaphorical symbol is significant for storytelling, the symbol will appear on most of the scenes. Therefore, we find groups of entities, which share similar lexical meanings and appear allover the movie. As a preliminary study, we have applied these approaches on terms in movie scripts with simple natural language processing techniques. Although we have not conducted evaluation yet, we anticipate that the proposed method will be helpful for analyzing artistic and topical intentions of movie directors. Copyright © by the paper’s authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

3. JAIHC

   A novel network virtualization based on data analytics in connected environment

   Khac-Hoai Nam Bui , Sungrae Cho , Jason J. Jung, Joongheon Kim , O-Joun Lee, and Woongsoo Na

   Journal of Ambient Intelligence and Humanized Computing, 2020

   Abs

   Big data analytics is a growing trend for network and service management. Some approaches such as statistical analysis, data mining and machine learning have become promising techniques to improve operations and management of information technology systems and networks. In this paper, we introduce a novel approach for network management in terms of abnormality detection based on data analytics. Particularly, the main research focuses on how the network configuration can be automatically and adaptively decided, given various dynamic contexts (e.g., network interference, heterogeneity and so on). Specifically, we design a context-based data-driven framework for network operation in connected environment which includes three layer architecture: (i) network entity layer; (ii) complex semantic analytics layer and (iii) action provisioning layer. A case study on interference-based abnormal detection for connected vehicle explains more detail about our work. © 2018, Springer-Verlag GmbH Germany, part of Springer Nature.

4. AIJ

   Story embedding: Learning distributed representations of stories based on character networks

   O-Joun Lee, and Jason J. Jung

   Artificial Intelligence, 2020

   Abs

   This study aims to learn representations of stories in narrative works (i.e., creative works that contain stories) using fixed-length vectors. Vector representations of stories enable us to compare narrative works regardless of their media or formats. To computationally represent stories, we focus on social networks among characters (character networks). We assume that the structural features of the character networks reflect the characteristics of stories. By extending substructure-based graph embedding models, we propose models to learn distributed representations of character networks in stories. The proposed models consist of three parts: (i) discovering substructures of character networks, (ii) embedding each substructure (Char2Vec), and (iii) learning vector representations of each character network (Story2Vec). We find substructures around each character in multiple scales based on proximity between characters. We suppose that a character’s substructures signify its ‘social roles’. Subsequently, a Char2Vec model is designed to embed a social role based on co-occurred social roles. Since character networks are dynamic social networks that temporally evolve, we use temporal changes and adjacency of social roles to determine their co-occurrence. Finally, Story2Vec models predict occurrences of social roles in each story for embedding the story. To predict the occurrences, we apply two approaches: (i) considering temporal changes in social roles as with the Char2Vec model and (ii) focusing on the final social roles of each character. We call the embedding model with the first approach ‘flow-oriented Story2Vec.’ This approach can reflect the context and flow of stories if the dynamics of character networks is well understood. Second, based on the final states of social roles, we can emphasize the denouement of stories, which is an overview of the static structure of the character networks. We name this model as ‘denouement-oriented Story2Vec.’ In addition, we suggest ‘unified Story2Vec’ as a combination of these two models. We evaluated the quality of vector representations generated by the proposed embedding models using movies in the real world. © 2020 Elsevier B.V.

5. Cancers

   Classification of breast cancer cells using the integration of high-frequency single-beam acoustic tweezers and convolutional neural networks

   Hae Gyun Lim, O-Joun Lee, K. Kirk Shung , Jin-Taek Kim , and Hyung Ham Kim

   Cancers, 2020

   Abs

   Single-beam acoustic tweezers (SBAT) is a widely used trapping technique to manipulate microscopic particles or cells. Recently, the characterization of a single cancer cell using high-frequency (>30 MHz) SBAT has been reported to determine its invasiveness and metastatic potential. Investigation of cell elasticity and invasiveness is based on the deformability of cells under SBAT’s radiation forces, and in general, more physically deformed cells exhibit higher levels of invasiveness and therefore higher metastatic potential. However, previous imaging analysis to determine substantial differences in cell deformation, where the SBAT is turned ON or OFF, relies on the subjective observation that may vary and requires follow-up evaluations from experts. In this study, we propose an automatic and reliable cancer cell classification method based on SBAT and a convolutional neural network (CNN), which provides objective and accurate quantitative measurement results. We used a custom-designed 50 MHz SBAT transducer to obtain a series of images of deformed human breast cancer cells. CNN-based classification methods with data augmentation applied to collected images determined and validated the metastatic potential of cancer cells. As a result, with the selected optimizers, precision, and recall of the model were found to be greater than 0.95, which highly validates the classification performance of our integrated method. CNN-guided cancer cell deformation analysis using SBAT may be a promising alternative to current histological image analysis, and this pretrained model will significantly reduce the evaluation time for a larger population of cells. © 2020 by the authors. Licensee MDPI, Basel, Switzerland.

6. ICEA

   Towards Story-based Summarization of Narrative Multimedia

   O-Joun Lee, Jin-Taek Kim , and Eun-Soon You

   2020

   Abs

   This study aims at summarizing narrative works (i.e., creative works that contain stories) in the consideration of their stories and types of required summaries. Various methods for story-based summarization have been proposed as a practical application of the character network analysis (i.e., a social network among characters that appeared in a story). However, the existing methods do not consider that summaries have different requirements according to their types (e.g., trailers, highlights, and recaps). These methods consist of three parts: (i) discretizing narrative works into regular units (e.g., scenes or shots), (ii) measuring the narrative significance of each unit, and (iii) generating summaries based on the narrative significance. Most of the existing studies have proposed their unique significance measurements based on individual narrative features. Also, since these methods have not considered the diverse types of summaries, they have simply selected top-N narrative units according to the measurements. In this study, we first introduce and redefine the narrative significance measurements. Subsequently, we propose a method for summarizing a narrative work regarding the requirements of the summaries by integrating the various significance measurements. © 2020 ACM.

7. Comp4Human

   Computing4Human 2020: The 1st international workshop on computational humanities and social sciences

   Jin-Taek Kim , Jason J. Jung, Eunsoon You , and O-Joun Lee

   2020

   Abs

   The 1st International Workshop on Computational Humanities and Social Sciences (Computing4Human) is hosted by Future IT Innovation Laboratory (i-Lab), Pohang University of Science and Technology (POSTECH), on the 15th of February 2020. This workshop tackled various research areas: the computational narrative analysis, the technology policies, and the human-computer interaction. Copyright © by the paper’s authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

8. ECIRw

   Measuring narrative fluency by analyzing dynamic interaction networks in textual narratives

   O-Joun Lee, and Jin-Taek Kim

   2020

   Abs

   This study aims to assess the fluency of narratives in textual multimedia (e.g., news articles, academic publications, novels, etc.). We measure the narrative fluency based on whether relationships between entities in the narrative (i.e., subjects and objects of events that compose the narrative) are consistently described with adequate rapidity. The relationships are represented by a dynamic interaction network (called ’entity network’), which has entities as nodes and co-occurrences between the entities as edges. Lack of consistency makes users confused about what the textual narratives want to present. If a narrative consistently concentrates on a topic or subject, its entity network will have few entities with high node centrality. Using consistency of the high centrality entities, we assess the fluency with three criteria: (i) consistency in each paragraph, (ii) consistency in the overall narrative, and (iii) consistency between the title and body. The rapidity of narrative development has to be appropriate for expected readers of the textual narratives. Too low rapidity causes redundancy, and high rapidity hinders the under-standability of the narratives. We assume structural changes in the entity network reflect the narrative rapidity. The structural change is measured by embedding structures of the entity network. Finally, we evaluated the effectiveness of the proposed methods using the editorials of the New York Times and human evaluators. Copyright © by the paper’s authors.

9. RACS

   Learning Multi-modal Representations of Narrative Multimedia: A Case Study of Webtoons

   O-Joun Lee, and Jin-Taek Kim

   2020

   Abs

   This study aims to learn task-agnostic representations of narrative multimedia. The existing studies focused on only stories in the narrative multimedia without considering their physical features. We propose a method for incorporating multi-modal features of the narrative multimedia into a unified vector representation. For narrative features, we embed character networks as with the existing studies. Textual features can be represented using the LSTM (Long-Short Term Memory) autoencoder. We apply the convolutional autoencoder to visual features. The convolutional autoencoder also can be used for the spectrograms of audible features. To combine these features, we propose two methods: early fusion and late fusion. The early fusion method composes representations of features on each scene. Then, we learn representations of a narrative work by predicting time-sequential changes in the features. The late fusion method concatenates feature vectors that are trained for allover the narrative work. Finally, we apply the proposed methods on webtoons (i.e., comics that are serially published through the web). The proposed methods have been evaluated by applying the vector representations to predicting the preferences of users for the webtoons. © 2020 ACM.

10. Sustain

    Discovering social desires and conflicts from subculture narrative multimedia

    O-Joun Lee, Heelim Hong , Eun-Soon You , and Jin-Taek Kim

    Sustainability, 2020

    Abs

    This study aims at discovering social desires and conflicts from subculture narrative multimedia. Since one of the primary purposes in the subculture consumption is vicarious satisfaction, the subculture works straightforwardly describe what their readers want to achieve and break down. The latent desires and conflicts are useful for understanding our society and realizing smart governance. To discover the social issues, we concentrate on that each subculture genre has a unique imaginary world that consists of inventive subjects. We suppose that the subjects correspond to individual social issues. For example, game fiction, one of the popular genres, describes a world like video games. Under game systems, everyone gets the same results for the same efforts, and it can be interpreted as critics for the social inequality issue. Therefore, we first extract subjects of genres and measure the membership degrees of subculture works for each genre. Using the subjects and membership degrees, we build a genealogy tree of subculture genres by tracing their evolution and differentiation. Then, we extract social issues by searching for the subjects that come from the real world, not imaginary. If a subculture work criticizes authoritarianism, it might include subjects such as government officials and bureaucrats. A combination of the social issues and genre genealogy tree will show diachronic changes in our society. We have evaluated the proposed methods by extracting social issues reflected in Korean web novels. © 2020 by the authors. Licensee MDPI, Basel, Switzerland.

11. IJCAIw

    IdeaNet: Potential opportunity discovery for business innovation

    O-Joun Lee, Sung Youn Park , and Jin-Taek Kim

    2020

    Abs

    This paper introduces an ongoing project for mining potential business opportunities from the existing business innovation cases. Crevate (a consulting company) has collected thousands of cases from news articles and columns that are in textual narratives. We aim to transform the cases into knowledge graphs that cover what kinds of ideas (e.g., untact) are applied to which industrial areas (e.g., cafe). By using link prediction methods, we will be able to evaluate the prominence of combinations (e.g., untact cafe) between ideas and domains. This study focuses on explaining the collected cases and knowledge graphs with running examples. © 2020 by the paper’s authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

12. Sensors

    Learning hierarchical representations of stories by using multi-layered structures in narrative multimedia

    O-Joun Lee, Jason J. Jung, and Jin-Taek Kim

    Sensors, 2020

    Abs

    Narrative works (e.g., novels and movies) consist of various utterances (e.g., scenes and episodes) with multi-layered structures. However, the existing studies aimed to embed only stories in a narrative work. By covering other granularity levels, we can easily compare narrative utterances that are coarser (e.g., movie series) or finer (e.g., scenes) than a narrative work. We apply the multi-layered structures on learning hierarchical representations of the narrative utterances. To represent coarser utterances, we consider adjacency and appearance of finer utterances in the coarser ones. For the movies, we suppose a four-layered structure (character roles ∈ characters ∈ scenes ∈ movies) and propose three learning methods bridging the layers: Char2Vec, Scene2Vec, and Hierarchical Story2Vec. Char2Vec represents a character by using dynamic changes in the character’s roles. To find the character roles, we use substructures of character networks (i.e., dynamic social networks of characters). A scene describes an event. Interactions between characters in the scene are designed to describe the event. Scene2Vec learns representations of a scene from interactions between characters in the scene. A story is a series of events. Meanings of the story are affected by order of the events as well as their content. Hierarchical Story2Vec uses sequential order of scenes to represent stories. The proposed model has been evaluated by estimating the similarity between narrative utterances in real movies. © 2020 by the authors. Licensee MDPI, Basel, Switzerland.

2019

1. FGCS

   Modeling affective character network for story analytics

   O-Joun Lee, and Jason J. Jung

   Future Generation Computer Systems, 2019

   Abs

   Consideration of the stories included in the narrative works is important for analyzing and providing narrative works (e.g., movies, novels, and comics) to users. In this study, we analyzed the stories in a narrative work with three goals: (i) eliciting, (ii) modeling, and (iii) utilizing the stories. Based upon our previous studies regarding ‘character networks’ (i.e., social networks among characters in the stories), we elicited the stories with three methods: (i) composing affective character networks with affective relationships among the characters, (ii) measuring temporal changes in tension according to the flows of the stories, and (iii) detecting affective events which refer to dramatic changes in the tension. The affective relationships contain emotional changes of the characters on each segment of the stories. By aggregating the characters’ emotional changes, we measured the tension of each segment. We called it ‘Affective Fluctuation’ and represented it as a discrete function (Affective Fluctuation Function, AFF). The AFFs enable us to detect affective events by using gradients of them and measure similarities among the stories by comparing their shapes. Also, we proposed a computational model of the stories by annotating the affective events and characters involved in those events. Finally, we demonstrated a practical application with a recommendation method which exploited the similarities between stories. Additionally, we verified the reliabilities and efficiencies of the proposed method for narrative works in the real world. © 2018 Elsevier B.V.

2. FDATA

   Is Performance of Scholars Correlated to Their Research Collaboration Patterns?

   Hyeon-Ju Jeon, O-Joun Lee, and Jason J. Jung

   Frontiers in Big Data, 2019

   Abs

   This study aims to validate whether the research performance of scholars correlates with how the scholars work together. Although the most straightforward approaches are centrality measurements or community detection, scholars mostly participate in multiple research groups and have different roles in each group. Thus, we concentrate on the subgraphs of co-authorship networks rooted in each scholar that cover (i) overlapping of the research groups on the scholar and (ii) roles of the scholar in the groups. This study calls the subgraphs “collaboration patterns” and applies subgraph embedding methods to discover and represent the collaboration patterns. Based on embedding the collaboration patterns, we have clustered scholars according to their collaboration styles. Then, we have examined whether scholars in each cluster have similar research performance, using the quantitative indicators. The coherence of the indicators cannot be solid proofs for validating the correlation between collaboration and performance. Nevertheless, the examination for clusters has exhibited that the collaboration patterns can reflect research styles of scholars. This information will enable us to predict the research performance more accurately since the research styles are more consistent and sustainable features of scholars than a few high-impact publications. Copyright © 2019 Jeon, Lee and Jung.

3. IPM

   Integrating character networks for extracting narratives from multimodal data

   O-Joun Lee, and Jason J. Jung

   Information Processing and Management, 2019

   Abs

   This study aims to integrate diverse data within narrative multimedia (i.e., artworks containing stories and distributed through multimedia) into a unified character network (i.e., a social network between characters that appear in the story). By combining multiple data sources (e.g., the text, video, and audio), we attempted to enhance the accuracy and semantic richness of existing character networks that confine themselves to a particular data source. To merge various data, we propose story synchronization for (i) improving the accuracy of data extracted from the narrative multimedia and (ii) integrating the data into the unified character network. The story synchronization mainly consists of three steps: synchronizing (i) scenes, (ii) characters, and (iii) character networks. First, we synchronize dialogues in the text and audio, to discover speakers and time of dialogues. This enables us to segment the scene using time periods when dialogues (in the text and audio) and characters (in the video) do not commonly occur. Through the scene segmentation, we can discretize stories in the narrative work. By comparing the occurrence of dialogues and characters in each scene, we synchronize identities of the characters in the text and video (e.g., names and faces of characters). Thereby, we can more accurately estimate participants and time of a conversation between characters (i.e., a set of connected dialogues). Based on the conversation, the existing character networks are refined and integrated into the unified character network. In addition, we verified the efficacy of the proposed methods using movies in the real world, which are among the most accessible and popular narrative multimedia. © 2019 Elsevier Ltd

4. WIMS

   Character network embedding-based plot structure discovery in narrative multimedia

   O-Joun Lee, and Jason J. Jung

   2019

   Abs

   This study aims to discover plot structures of narrative multimedia (i.e., creative works that contain stories and are distributed through multimedia). Although a few studies have been conducted to extract subplots from narrative works based on occurrences of characters, the subplots do not always have distinctive compositions of characters. Therefore, we propose a method for discriminating the subplots based on overall social relationships among characters. To understand ways how a story is interwoven with subplots, it is important to describe how story lines in each subplot are developed and how subplots are split and rearranged. As a first step, we propose a measurement for estimating rapidity of story development based on changes in the social relationships among characters. Subsequently, we suggest a model for representing plot structures based on transitions among subplots, the narrative significance of each scene, and the main characters of each subplot. We anticipate that the plot structures can be applied for summarizing narrative multimedia regarding the story. © 2019 ACM.

2018

1. IUIw

   Explainable movie recommendation systems by using story-based similarity

   O-Joun Lee, and Jason J. Jung

   2018

   Abs

   The goal of this paper is to provide a story-based explanation for movie recommendation systems, achieved by a multiaspect explanation and narrative analysis methods. We explain how and why particular movies are similar based on following two aspects: (i) composition of movie characters and (ii) interactions among the characters. These aspects correspond to story-based features of the movies that are extracted from character networks (i.e., social networks among the characters). By using the story-based features, we can explain the reason why two arbitrary movies are similar or not. We anticipate that the proposed method could improve the explainability of the recommender systems for movies. © 2018 Copyright for the individual papers remains with the authors.

2. BDTA

   Network engineering: Towards data-driven framework for network configuration

   Khac Hoai Nam Bui , Sungrae Cho , Jason J. Jung, Joongheon Kim , O-Joun Lee, and Woongsoo Na

   Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, 2018

   Abs

   In this paper, we want to introduce a new research area “network engineering”. The main research question is how the network configuration can be automatically and adaptively decided, given various dynamic contexts (e.g., network interference, heterogeneity and so on). The aim of this work is to design data-driven framework which is in three layer architecture (i.e., network entity layer, complex semantic analytics layer, and action provisioning layer). © ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2018.

3. MONET

   Owner-Borrower Model for Recommenders in O2O Services

   O-Joun Lee , and Jai E. Jung

   Mobile Networks and Applications, 2018

   Abs

   With remarkable successes of sharing economy services (e.g., UBER (https://www.uber.com), Airbnb (https://www.airbnb.com), and so on), the amount of items which are distributed through these services is rapidly increasing. Therefore recommender systems for the sharing economy services are required. However, the existing recommenders are hard to support the sharing economy services, since they have focused on a ‘Item-User’ model that the recommenders provide satisfiable items to consumers (users) in accordance with only the consumers’ preferences. In this regard, we suggest a novel recommendation model, ‘Owner-Borrower’ model which considers the preferences of both sides: owners and borrowers of properties (items). Also, we propose a recommendation method based on the proposed model by applying a tensor factorization method and the Gale-Shapley algorithm. The tensor factorization is used for estimating preferences of the owners and the borrowers. With the estimated preferences, the Gale-Shapley algorithm makes optimal matches between the borrowers and the owners’ properties. © 2018, Springer Science+Business Media, LLC, part of Springer Nature.

4. JUCS

   Multi-scaled spatial analytics on discovering latent social events for smart urban services

   O-Joun Lee, Yunhu Kim , Hoang Long Nguyen , and Jai E. Jung

   Journal of Universal Computer Science, 2018

   Abs

   The goal of this paper is to discover latent social events from social media for sensitively understanding social opinions that appeared within a city. The latent social event indicates a regional and inconspicuous social event which is mostly buried under macroscopic trends or issues. To detect the latent social event, we propose three methods: i) discovering areas-of-interest (AOIs), ii) allocating social texts to the AOIs, and iii) detecting social events in each AOI. The AOIs can be composed by grouping social texts which are topically and spatially homogeneous. To make the AOIs dynamic and incremental, we use windows for allocating a social text to an adequate AOI. Lastly, the latent social events are detected from the AOI on the basis of keywords and temporal distribution of the social texts. Although, in this study, we limited the proposed method into analyzing social media, it could be extended to detecting events among agents/things/sensors. © J.UCS.

5. CCPE

   Spatio-temporal query contextualization for microtext retrieval in social media

   Jae-Hong Park , O-Joun Lee , and Jai E. Jung

   Concurrency and Computation: Practice and Experience, 2018

   Abs

   Generally, texts on the social media (eg, Twitter and Facebook) are too short (microtexts) to understand the meaning and to search for relevant texts. It is difficult for the conventional information retrieval systems to conduct the searching tasks. Thereby, in this paper, we propose a novel approach on query contextualization by integrating all possible microtexts by considering spatio-temporal contexts. The proposed approach consists of two steps, which are (i) to understand and process microtexts in social media and (ii) to reformulate the queries for searching for relevant microtexts in these social media. To evaluate the performance of the query contextualization approach, microtexts from Twitter have been collected during 4 months. © 2018 John Wiley & Sons, Ltd.

6. ECIRw

   Measuring character-based story similarity by analyzing movie scripts

   O-Joun Lee, Nayoung Jo , and Jason J. Jung

   2018

   Abs

   The goal of this paper is to measure similarity among the stories for categorizing movies. Although genres are well-performing as movies’ categories, users have difficulty for predicting substances of the movies through the genres. Therefore, we proposed the story-based taxonomy of the movies and a method for constructing it automatically. In order to reflect characteristics of the stories, we used two kinds of features: (i) proximity among movie characters and (ii) genres of the movies. Based on the features, we constructed the story-based taxonomy by clustering the movies. We anticipate that the proposed taxonomy could make the users imagine and predict substances of movies through comprehending which movies contain similar stories. Copyright © 2018 for the individual papers by the paper’s authors.

2017

1. IEEE Access

   Towards Ontological Approach on Trust-Aware Ambient Services

   O-Joun Lee, Hoang Long Nguyen , Jai E. Jung , Tai-Won Um , and Hyun-Woo Lee

   IEEE Access, 2017

   Abs

   With various information sources (e.g., from IoT sensors to social media), it is difficult to provide users with trustworthy services in ambient environment. The aim of this paper is i ) to design trust ontology for representing semantics of the ambient services and ii ) to compute trust measures among users by using a personalized trust ontology. In particular, given a large amount of data collected from ambient sensors, efficient trust computation and reasoning are required for the stability and reliability. Thereby, we propose trust ontology-based framework for deriving personalized ontologies for individual users according to their preference, perspective, and purpose. To evaluate the proposed model, we have figured out a method how the degree of trust is estimated based on the trust ontology. Furthermore, we have proved that the proposed method is reliable with a case study on a social media (Twitter) for a particular domain (restaurant). © 2013 IEEE.

2. FGCS

   Sequence Clustering-based Automated Rule Generation for Adaptive Complex Event Processing

   O-Joun Lee , and Jai E. Jung

   Future Generation Computer Systems, 2017

   Abs

   In Complex Event Processing (CEP), complex events are detected according to a set of rules that are defined by domain experts. However, it makes the reliability of the system decreased as dynamic changes occur in the domain environment or domain experts make mistakes. To address such problem, this study proposes a Sequence Clustering-based Automated Rule Generation (SCARG) that can automatically generate rules by mining decision-making history of domain experts based on sequence clustering and probabilistic graphical modeling. Furthermore, based on a two-way learning approach, the proposed method is able to support automated regular or occasional rule updates. It makes self-adaptive CEP system possible by combining the rule generation method and the existing dynamic CEP systems. This technique is verified by establishing an automated stock trading system, and the performance of the system is measured in terms of the rate of return. The study solves the aforementioned problems and shows excellent results with an increase of 19.32% in performance when compared to the existing dynamic CEP technique. © 2016 Elsevier B.V.

3. MTAP

   A computational model of transmedia ecosystem for story-based contents

   Jai E. Jung , O-Joun Lee, Eun-Soon You , and Myoung-Hee Nam

   Multimedia Tools and Applications, 2017

   Abs

   Story-based contents (e.g., novel, movies, and computer games) have been dynamically transformed into various media. In this environment, the contents are not complete in themselves, but closely connected with each other. Also, they are not simply transformed form a medium to other media, but expanding their stories. It is called as a transmedia storytelling, and a group of contents following it is called as a transmedia ecosystem. Since the contents are highly connected in terms of the story in the transmedia ecosystem, the existing content analysis methods are hard to extract relationships between the contents. Therefore, a proper content analysis method is needed with considering expansions of the story. The aim of this work is to understand how (and why) such contents are transformed by i) defining the main features of the transmedia storytelling and ii) building the taxonomy among the transmedia patterns. More importantly, computational transmedia ecosystem is designed to process a large number of the contents, and to support high understandability of the complex transmedia patterns. © 2016, Springer Science+Business Media New York.

4. BDTA

   A novel method for extracting dynamic character network from movie

   Quang Dieu Tran , Dosam Hwang , O-Joun Lee, and Jason J. Jung

   Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, 2017

   Abs

   In this decade, the number of movies is increasing rapidly. Many studies have been proposed to assist users in movie understanding. In which, these methods are taken into account movie content analysis using social network for discovering relationships among characters and so on. However, these methods have shown some unsatisfactorily in dynamic changing of multimedia contents such as the character’s relationships over time. For overcoming this issue, we proposed a novel method for extracting dynamic character network from a movie. © ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2017.

5. IEEE Access

   Event-Driven Trust Refreshment on Ambient Services

   Hoang Long Nguyen , O-Joun Lee , Jai E. Jung , Jaehwa Park , Tai-Won Um , and Hyun-Woo Lee

   IEEE Access, 2017

   Abs

   Since trust among entities can change according to various conditions, it is necessary for ambient services to determine when and how the trust has to be updated. Therefore, our contribution in this paper is to present: 1) a new definition of trust that can be extended to various domains; 2) a novel method based on social events and patterns to trigger trust refreshment in ambient services; and 3) a web application framework (called SocioScope) for collecting and analyzing data from multiple data sources. Finally, the case study suggests that this proposal could be applied to trust-aware ambient and recommendation systems. © 2013 IEEE.

6. DaMIS

   Spatio-Temporal Contextualization of Queries for Microtexts in Social Media: Mathematical Modeling

   Jae-Hong Park , O-Joun Lee, Joo-Man Han , Eon-Ji Lee , Jason J. Jung, Luca Carratore , and Francesco Piccialli

   2017

   Abs

   In this paper, we present our ongoing project on query contextualization by integrating all possible IoT-based data sources. Most importantly, mobile users are regarded as the IoT sensors which can be the textual data sources with spatio-temporal contexts. Given a large amount of text streams, it has been difficult for the traditional information retrieval systems to conduct the searching tasks. The goal of this work is i) to understand and process microtexts in social media (e.g., Twitter and Facebook), and ii) to reformulate the queries for searching for relevant microtexts in these social media. Peer-review under responsibility of the Conference Program Chairs. © 2017 The Authors. Published by Elsevier B.V.

7. IE

   Towards affective lifelogging with information fusion

   Jason J. Jung, Minsung Hong , O-Joun Lee, Jaehong Park , and Chang Choi

   2017

   Abs

   Recently, most of context-aware services are trying to exploit the emotional contexts of the target users. The aim of this conceptual paper is to discuss affective lifelogging framework which can recognize the emotions by integrating multimodal information from multiple sources. Moreover, we will mention the open problems on affective lifelogging. © 2017 IEEE.

8. MTAP

   Exploiting character networks for movie summarization

   Quang Dieu Tran , Dosam Hwang , O-Joun Lee , and Jai E. Jung

   Multimedia Tools and Applications, 2017

   Abs

   Movie summarization focuses on providing as much information as possible for shorter movie clips while still keeping the content of the original movie and presenting a faster way for the audience to understand the movie. In this paper, we propose a novel method to summarize a movie based on character network analysis and the appearance of protagonist and main characters in the movie. Experiments were carried out for 2 movies (Titanic (1997) and Frozen (2013)) to show that our method outperforms conventional approaches in terms of the movie summarization rate. © 2016, Springer Science+Business Media New York.

2016

1. APH

   Adaptive collaborative filtering based on scalable clustering for big recommender systems

   O-Joun Lee, Min-Sung Hong , Jason J. Jung, Juhyun Shin , and Pankoo Kim

   Acta Polytechnica Hungarica, 2016

   Abs

   The large amount of information that is currently being collected (the so-called “big data”), have resulted in model-based Collaborative Filtering (CF) methods to encountering limitations, e.g., the sparsity problem and the scalability problem. It is difficult for model-based CF methods to address the scalability-performance trade-off. Therefore, we propose a scalable clustering-based CF method in this paper that can help provide a balance by re-locating elements in the cluster model. The proposed method is evaluated by performing a comparison against existing methods in terms of measurements for the Mean Absolute Error (MAE) and response time to assess the performance and scalability. The experimental results show that the proposed method improves the MAE and the response time by 50.79% and 48.25%, respectively. © 2016, Budapest Tech Polytechnical Institution. All rights reserved.

2. ICCASA

   Context-based traffic recommendation system

   Khac-Hoai Nam Bui , Xuan Hau Pham , Jason J. Jung, O-Joun Lee, and Min-Sung Hong

   Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, 2016

   Abs

   In this paper, we propose a new traffic system recommendation based on support real-time flows in highly unpredictable sensor network environments. The approach system is real-time recommendation system which meet various demands of users. The proposed algorithm include two phases. First phase is proposed to deal with the real-time problem. By this way, the drivers are able to transfer on the way with the shortest-time. For second phase, a research algorithm based on Depth First Search (DFS) algorithm will recommend the paths which meet demands of drivers based their context such as the paths with include the famous landscapes or the paths where they can find out good restaurants for their break while driving. © ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2016.

3. AfCAI

   Affective character network for understanding plots of narrative multimedia contents

   O-Joun Lee, and Jason J. Jung

   2016

   Abs

   It is important to understand the stories from narrative multimedia contents (e.g., movies), and to exploit the stories for smart services (e.g., video summarization and personalized multimedia recommendation). In this paper, we extend CharNet (Character Network) to Affective CharNet (Affective Character Network) by annotating emotional relationships between characters. More importantly, we propose a novel method on analyzing Affective CharNet for extracting the plots from the narrative multimedia contents. © 2016, CEUR-WS. All rights reserved.

4. ISAmI

   Dynamic traffic light control system based on process synchronization among connected vehicles

   Khac-Hoai Nam Bui , O-Joun Lee, Jason J. Jung, and David Camacho

   Advances in Intelligent Systems and Computing, 2016

   Abs

   Vehicular traffic is tremendously increasing around the world, especially in large urban areas. The resulting congestion has become a key issue and emerging research topic to transportation specialist and decision makers. In this study, inspired by recent advanced vehicle technologies, we take into account in improving traffic flow in real-time problem. In order to solve the problem, we propose a new approach to manage traffic flow at the intersection in real-time via controlling by traffic light scheduling. In particular, the proposed method is based on process synchronization theory and connected vehicle technology where each vehicle is able to communicate with others. The traffic deadlock is also taken into consideration in case of high traffic volume. The simulation shows the potential results comparing with the existing traffic management system. © Springer International Publishing Switzerland 2016.

5. ICCASA

   MBTI-based collaborative recommendation system: A case study of Webtoon contents

   Myeong-Yeon Yi , O-Joun Lee, and Jason J. Jung

   Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, 2016

   Abs

   A large number of Webtoon contents has caused difficulties on finding relevant Webtoons for users. Thereby, an efficient recommendation services are needed. However, since the existing recommendation method (e.g. collaborative filtering) has two fundamental problems: (i.e., data sparsity and scalability problem), it has difficulties with reflecting users’ personality. In this paper, we propose the MBTI-CF method to solve these problems and to involve users’ personality by building personality-based neighborhood using MBTI. In order to verify the efficiency of the proposed method, we conducted statistical testing by user survey (anonymous users have rated set of the pre-selected Webtoon contents). Three experimental results have shown that MBTI-CF provides improvement in terms of the data sparsity problem and the scalability problem and offers more stable performance. © ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2016.

2015

1. ACIIDS

   Adaptive complex event processing based on collaborative rule mining engine

   O-Joun Lee, Eunsoon You , Min-Sung Hong , and Jason J. Jung

   Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2015

   Abs

   Complex Event Processing (CEP) detects complex events or patterns of event sequences based on a set of rules defined by a domain expert. However, it lowers the reliability of a system as the set of rules defined by an expert changes along with dynamic changes in the domain environment. A human error made by an expert is another factor that may undermine the reliability of the system. In an effort to address such problems, this study introduces Collaborative Rule Mining Engine (CRME) designed to automatically mine rules based on the history of decisions made by a domain expert by adopting a collaborative filtering approach, which is effective in mimicking and predicting human decision-making in an environment where there are sufficient data or information to do so. Furthermore, this study suggests an adaptive CEP technique, which does not hamper the reliability since it prevents potential errors caused by mistakes of domain experts and adapts to changes in the domain environment on its own as it is linked to the system proposed by Bharagavi [10]. In a bid to verify this technique, an automated stocks trading system will be established and its performance will be measured using the rate of return. © Springer International Publishing Switzerland 2015.

2. CYBCONF

   Predictive Clustering for performance stability in collaborative filtering techniques

   O-Joun Lee, Jason J. Jung, and You Eunsoon

   2015

   Abs

   Model-based collaborative filtering improves the fundamental limitations of the collaborative filtering facing the issues of data sparsity and scalability while presenting other constraints of high costs of model building and the tradeoff between performance and scalability. Such tradeoff results in reduced coverage, which is one sort of the sparsity issue. Furthermore, high model building costs lead to unstable performance driven by cumulative changes in the domain environment. To solve these problems, we propose Predictive Clustering-based CF (PCCF) that incorporates the Markov model and fuzzy clustering with Clustering based CF (CBCF). The method improves performance instability by tracking the changes in user preferences and bridging the gap between the static model and dynamic users. Furthermore, the issue of reduced coverage is also improved by expanding the coverage based on transition probabilities. The proposed method has been validated by testing the robustness of performance instability and scalability-performance tradeoff. In comparison with the existing techniques, the suggested method shows slight performance improvement. Notwithstanding, it is more advanced than the existing techniques in terms of the range that indicates the level of performance fluctuation. This signifies that the proposed method, despite the slight performance improvement, clearly offers better performance stability compared to the existing techniques. © 2015 IEEE.

2014

1. Inform

   Meta-data configuration and learning technique for wellness content recommendation

   Min-Sung Hong , O-Joun Lee, Won-Jin Lee , Kee-Won Kim , Seung-Hoon Kim , Hee-Jung Han , and Jae-Dong Lee

   Information, 2014

   Abs

   The method for personalized wellness-content recommendation has actively studied in the field of information technology convergence. But a problem with low reliability of recommendations has emerged. Because existing studies deal with only one or two areas of wellness, to solve the reliability problems, a study is needed into an integration technique that can manage several aspects of wellness. In this paper, we propose a metadata configuration and feature analysis technique for integrated management in areas of wellness. And we propose a learning technique for analyzed wellness features using user feedback to increase the reliability of user recommendations. To verify the proposed technique, we compared the results of a pre-survey with those from the proposed technique. © 2014 International Information Institute.