Skip to main content

Data Science

The hub for all things related to data analysis and data science.

  • 9 Topics
  • 0 Replies
9 Topics
Sothan
SothanCommunity Manager
published in Data Science
Setting Up a Windows Environment for Data Science: How Benedict Neo Configured His Workstation

As a Z by HP Ambassadors, Benedict Neo was equipped with the HP Z6 G5 Tower Workstation, featuring an AMD Ryzen Threadripper PRO 7965WX processor and NVIDIA RTX 6000 graphics. This powerful setup is tailored to significantly enhance his data science projects. However, a new workstation means he needed to establish a fresh data science environment.Here’s a walkthrough into how he configured his workstation, along with valuable tips to help you optimize your own setup for maximum efficiency. In this setup, Benedict walks you through installing PowerShell 7, WSL, Ubuntu, Git, setting up CUDA, and installing Python with Mamba.How to Setup Windows for Data Science HP Z6 G5 SpecsProcessor: AMD Ryzen Threadripper PRO 7965WX 24-Cores 4.20 GHz Installed RAM: 256 GB (255 GB usable) System type: 64-bit operating system, x64-based processor Graphics: NVIDIA RTX 6000 Ada 48 GB 4DP Graphics

120
Sothan
1 month ago
jrgosalvez
jrgosalvezCommunity Manager
posted in Data Science
Entropy Based Minority Oversampling (E-SMOTE) for Imbalanced Dataset Samples

This week Anbu Valluvan Devadasan et all published a paper titled “E-SMOTE: Entropy Based Minority Oversampling for Heart Failure and AIDS Clinical Trails Analysis.”I reached out to Anbu to chat about the Entropy-based Synthetic Minority Oversampling Technique (SMOTE) which uses a subset of the dataset from the minority class to address overbias of a majority class. Anbu explained that the team used entropy as a metric to assess influence of a minority class on outcomes and outperform conventional SMOTE oversampling techniques in some cases. Results illustrated that both SMOTE and E-SMOTE techniques should be considered for addressing imbalanced data samples as they can be used to address class imbalance. Anbu is excited and looking forward to dig deeper into which data characteristics affect metrics and how.Anbu’s open to chat about his work. If you’re interested, ping him - linkedin.com/in/anbuvalluvan. S. Veerla, A. V. Devadasan, M. Masum, M. Chowdhury and H. Shahriar, "E-SMOTE: Ent

310
jrgosalvez
2 months ago
jrgosalvez
jrgosalvezCommunity Manager
posted in Data Science
Automatic segmentation and classification of Papanicolaou-stained cells

Maikel Ronnau et al will publish Automatic segmentation and classification of Papanicolaou-stained cells and dataset for oral cancer detection in next month’s Computers in Biology and Medicine.Their research uses a CNN U-Net architecture with ResNet encoder to achieve expert level performance by segmenting and classifying a trained on UFSC OCPap dataset of oral cytology images. The team uses transfer learning for image segmentation to train the model which is used to evaluate more than 1500 images from 52 patients labeled by specialists resulting in an average Dice score of 0.66 and Intersection over Union (IoU) of 0.65. “The CNN model architecture for encoding layers are based on DenseNet-169 while the decoding layers are LinkNet but to replace the regular softmax layer with a temperature scaling softmax with a temperature parameter value of 0.1 to increase the confidence of the predictions and avoid the bias towards the prediction of background pixels. The model’s prediction is furth

260
jrgosalvez
3 months ago
jrgosalvez
jrgosalvezCommunity Manager
posted in Data Science
Mirage, a RL Slurm-compatible resource provisioner for GPU clusters

GPU availability and job interruption can be an issue for me, so this paper caught my attention. The paper is titled “Mirage: Towards Low-interruption Services on Batch GPU Clusters with Reinforcement Learning” and it proposes a method for reducing service interruptions on GPU clusters for deep learning jobs.The tool is called Mirage. Mirage is a Slurm-compatible foundational model that uses statistical and reinforcement learning (RL) techniques to proactively provision resources to mitigate interruptions. To accomplish this, researchers alter the decoder transformer architecture to create a Mixture of Experts (MoE) of neural networks. The neural networks share the same network architecture to support scaling up of model capacity and determine ‘best-fit experts’ that optimize job trace analysis outputs for resource allocations. What’s really cool about this paper is how it investigates the use of Deep Q-Learning RL techniques to reenforce rewards to improve GPU queue wait times for a f

30
jrgosalvez
3 months ago
jrgosalvez
jrgosalvezCommunity Manager
posted in Data Science
NOVA “Secrets in Your Data” - can a decentralized internet get legs?

NOVA on PBS has a special titled “Secrets in Your Data” where they talk about privacy, access, use, and innovations. A fun show to watch for fellow data lovers… and  if you’re also a fan of the HBO show “Silicon Valley” like I am, you’ll enjoy the discussion around a decentralized internet… will it take off?NOVA - Secrets In Your Data

50
jrgosalvez
5 months ago
Sothan
SothanCommunity Manager
posted in Data Science
Ken Jee Interviews HP's Head of Product Marketing Andrew Kemp

Today, I had the pleasure of interviewing Andrew Kemp. Andrew is the head of product marketing for data science and AI at Z by HP. Andrew recruited me into the Z by HP global ambassador program and is responsible for its creation. In this episode, we learn about his experience teaching in North Korea for 4 years as well as how the landscape of hardware and community is changing.  

260
Sothan
5 months ago
jrgosalvez
jrgosalvezCommunity Manager
asked in Data Science
Do you ever need to keep data out of the cloud?

Is there ever a scenario you need to keep data out of the cloud? For me it’s company data we’re looking to retain as IP. I’m curious to hear what types of data others consider sensitive.

310
jrgosalvez
5 months ago
jrgosalvez
jrgosalvezCommunity Manager
published in Data Science
We surveyed Data Scientists and ITDMs about Linux and CLI. Here's what they said...Survey

We did a quick survey to more than 200 Data Scientists and ITDMs. Sharing some interesting responses about tools used to create AI. 

150
jrgosalvez
6 months ago
matt.trainor
Explorer
matt.trainorExplorer
posted in Data Science
DS Joke of the Day

Leaving this joke here and hopes that it brightens your day:There are two types of Data Scientists: 1. Those that can extrapolate from incomplete data

220
matt.trainor
Explorer
1 year ago
Terms & ConditionsCookie settings