Skip to main content

Data Science

The hub for all things related to data analysis and data science.

  • 8 Topics
  • 0 Replies
8 Topics
jrgosalvez
jrgosalvezCommunity Manager
published in Data Science
We surveyed Data Scientists and ITDMs about Linux and CLI. Here's what they said...Survey

We did a quick survey to more than 200 Data Scientists and ITDMs. Sharing some interesting responses about tools used to create AI. 

110
jrgosalvez
3 months ago
jrgosalvez
jrgosalvezCommunity Manager
posted in Data Science
Entropy Based Minority Oversampling (E-SMOTE) for Imbalanced Dataset Samples

This week Anbu Valluvan Devadasan et all published a paper titled “E-SMOTE: Entropy Based Minority Oversampling for Heart Failure and AIDS Clinical Trails Analysis.”I reached out to Anbu to chat about the Entropy-based Synthetic Minority Oversampling Technique (SMOTE) which uses a subset of the dataset from the minority class to address overbias of a majority class. Anbu explained that the team used entropy as a metric to assess influence of a minority class on outcomes and outperform conventional SMOTE oversampling techniques in some cases. Results illustrated that both SMOTE and E-SMOTE techniques should be considered for addressing imbalanced data samples as they can be used to address class imbalance. Anbu is excited and looking forward to dig deeper into which data characteristics affect metrics and how.Anbu’s open to chat about his work. If you’re interested, ping him - linkedin.com/in/anbuvalluvan. S. Veerla, A. V. Devadasan, M. Masum, M. Chowdhury and H. Shahriar, "E-SMOTE: Ent

0
jrgosalvez
1 day ago
jrgosalvez
jrgosalvezCommunity Manager
posted in Data Science
Automatic segmentation and classification of Papanicolaou-stained cells

Maikel Ronnau et al will publish Automatic segmentation and classification of Papanicolaou-stained cells and dataset for oral cancer detection in next month’s Computers in Biology and Medicine.Their research uses a CNN U-Net architecture with ResNet encoder to achieve expert level performance by segmenting and classifying a trained on UFSC OCPap dataset of oral cytology images. The team uses transfer learning for image segmentation to train the model which is used to evaluate more than 1500 images from 52 patients labeled by specialists resulting in an average Dice score of 0.66 and Intersection over Union (IoU) of 0.65. “The CNN model architecture for encoding layers are based on DenseNet-169 while the decoding layers are LinkNet but to replace the regular softmax layer with a temperature scaling softmax with a temperature parameter value of 0.1 to increase the confidence of the predictions and avoid the bias towards the prediction of background pixels. The model’s prediction is furth

0
jrgosalvez
1 month ago
jrgosalvez
jrgosalvezCommunity Manager
posted in Data Science
Mirage, a RL Slurm-compatible resource provisioner for GPU clusters

GPU availability and job interruption can be an issue for me, so this paper caught my attention. The paper is titled “Mirage: Towards Low-interruption Services on Batch GPU Clusters with Reinforcement Learning” and it proposes a method for reducing service interruptions on GPU clusters for deep learning jobs.The tool is called Mirage. Mirage is a Slurm-compatible foundational model that uses statistical and reinforcement learning (RL) techniques to proactively provision resources to mitigate interruptions. To accomplish this, researchers alter the decoder transformer architecture to create a Mixture of Experts (MoE) of neural networks. The neural networks share the same network architecture to support scaling up of model capacity and determine ‘best-fit experts’ that optimize job trace analysis outputs for resource allocations. What’s really cool about this paper is how it investigates the use of Deep Q-Learning RL techniques to reenforce rewards to improve GPU queue wait times for a f

0
jrgosalvez
1 month ago
jrgosalvez
jrgosalvezCommunity Manager
posted in Data Science
NOVA “Secrets in Your Data” - can a decentralized internet get legs?

NOVA on PBS has a special titled “Secrets in Your Data” where they talk about privacy, access, use, and innovations. A fun show to watch for fellow data lovers… and  if you’re also a fan of the HBO show “Silicon Valley” like I am, you’ll enjoy the discussion around a decentralized internet… will it take off?NOVA - Secrets In Your Data

0
jrgosalvez
2 months ago
Sothan
SothanCommunity Manager
posted in Data Science
Ken Jee Interviews HP's Head of Product Marketing Andrew Kemp

Today, I had the pleasure of interviewing Andrew Kemp. Andrew is the head of product marketing for data science and AI at Z by HP. Andrew recruited me into the Z by HP global ambassador program and is responsible for its creation. In this episode, we learn about his experience teaching in North Korea for 4 years as well as how the landscape of hardware and community is changing.  

0
Sothan
3 months ago
jrgosalvez
jrgosalvezCommunity Manager
asked in Data Science
Do you ever need to keep data out of the cloud?

Is there ever a scenario you need to keep data out of the cloud? For me it’s company data we’re looking to retain as IP. I’m curious to hear what types of data others consider sensitive.

0
jrgosalvez
3 months ago
matt.trainor
Explorer
matt.trainorExplorer
posted in Data Science
DS Joke of the Day

Leaving this joke here and hopes that it brightens your day:There are two types of Data Scientists: 1. Those that can extrapolate from incomplete data

0
matt.trainor
Explorer
10 months ago
Terms & ConditionsCookie settings