FAKE NEWS¶

Assignment brief¶

This dataset contains information on a number of social media posts and classifies them as real or fake news. Example insight: Can you build a machine learning model to detect which posts are likely to be fake news?

File name: news_articles.csv

Find out more about this dataset here

Extension: To extend the project, try using two different supervised learning algorithms and use methods to measure their accuracy. Which model would you choose? Why?

In [2]:

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from sklearn.linear_model import LinearRegression
from pandas_profiling import ProfileReport

In [4]:

news = pd.read_csv('news_articles.csv')
news.head()

Out[4]:

	author	published	title	text	language	site_url	main_img_url	type	label	title_without_stopwords	text_without_stopwords	hasImage
0	Barracuda Brigade	2016-10-26T21:41:00.000+03:00	muslims busted they stole millions in govt ben...	print they should pay all the back all the mon...	english	100percentfedup.com	http://bb4sp.com/wp-content/uploads/2016/10/Fu...	bias	Real	muslims busted stole millions govt benefits	print pay back money plus interest entire fami...	1.0
1	reasoning with facts	2016-10-29T08:47:11.259+03:00	re why did attorney general loretta lynch plea...	why did attorney general loretta lynch plead t...	english	100percentfedup.com	http://bb4sp.com/wp-content/uploads/2016/10/Fu...	bias	Real	attorney general loretta lynch plead fifth	attorney general loretta lynch plead fifth bar...	1.0
2	Barracuda Brigade	2016-10-31T01:41:49.479+02:00	breaking weiner cooperating with fbi on hillar...	red state \nfox news sunday reported this mor...	english	100percentfedup.com	http://bb4sp.com/wp-content/uploads/2016/10/Fu...	bias	Real	breaking weiner cooperating fbi hillary email ...	red state fox news sunday reported morning ant...	1.0
3	Fed Up	2016-11-01T05:22:00.000+02:00	pin drop speech by father of daughter kidnappe...	email kayla mueller was a prisoner and torture...	english	100percentfedup.com	http://100percentfedup.com/wp-content/uploads/...	bias	Real	pin drop speech father daughter kidnapped kill...	email kayla mueller prisoner tortured isis cha...	1.0
4	Fed Up	2016-11-01T21:56:00.000+02:00	fantastic trumps point plan to reform healthc...	email healthcare reform to make america great ...	english	100percentfedup.com	http://100percentfedup.com/wp-content/uploads/...	bias	Real	fantastic trumps point plan reform healthcare ...	email healthcare reform make america great sin...	1.0

In [5]:

news_profile = ProfileReport(news, title="Country Data Profiling Report", explorative=True)

In [6]:

news_profile.to_notebook_iframe()

Summarize dataset: 100%|██████████| 27/27 [00:09<00:00,  2.97it/s, Completed]                                
Generate report structure: 100%|██████████| 1/1 [00:02<00:00,  2.94s/it]
Render HTML: 100%|██████████| 1/1 [00:00<00:00,  1.45it/s]

Feature Name	Definition	Data Type	Cardinality
author	Name of the author.	String, categorical	High
published	Date of publication.	Datetime, categorical	High
title	Title of the article.	String, categorical	High
text	Content of the article.	String, categorical	High
language	The language the article was written in.	String, categorical
site_url	The website URL for the article.	String (or char), categorical	High
main_img_url	The URL for the main image used.	String (or char), categorical	High
type	The type of article written (genre).	String, categorical
label	Whether the article is real or fake (target feature).	Boolean, categorical
title_without_stopwords	The title of the article (excluding stop words such as: its, an, the, for, and that).	String, categorical	High
text_without_stopwords	The content of the article.	String, categorical	High
hasImage	Whether the article has an image or not.	Boolean, categorical

In [3]:

news.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 2096 entries, 0 to 2095
Data columns (total 12 columns):
 #   Column                   Non-Null Count  Dtype  
---  ------                   --------------  -----  
 0   author                   2096 non-null   object 
 1   published                2096 non-null   object 
 2   title                    2096 non-null   object 
 3   text                     2050 non-null   object 
 4   language                 2095 non-null   object 
 5   site_url                 2095 non-null   object 
 6   main_img_url             2095 non-null   object 
 7   type                     2095 non-null   object 
 8   label                    2095 non-null   object 
 9   title_without_stopwords  2094 non-null   object 
 10  text_without_stopwords   2046 non-null   object 
 11  hasImage                 2095 non-null   float64
dtypes: float64(1), object(11)
memory usage: 196.6+ KB

In [4]:

news.isnull().sum()

Out[4]:

author                      0
published                   0
title                       0
text                       46
language                    1
site_url                    1
main_img_url                1
type                        1
label                       1
title_without_stopwords     2
text_without_stopwords     50
hasImage                    1
dtype: int64

In [5]:

news.isnull()

Out[5]:

	author	published	title	text	language	site_url	main_img_url	type	label	title_without_stopwords	text_without_stopwords	hasImage
0	False	False	False	False	False	False	False	False	False	False	False	False
1	False	False	False	False	False	False	False	False	False	False	False	False
2	False	False	False	False	False	False	False	False	False	False	False	False
3	False	False	False	False	False	False	False	False	False	False	False	False
4	False	False	False	False	False	False	False	False	False	False	False	False
...	...	...	...	...	...	...	...	...	...	...	...	...
2091	False	False	False	True	False	False	False	False	False	False	True	False
2092	False	False	False	True	False	False	False	False	False	False	True	False
2093	False	False	False	True	False	False	False	False	False	False	True	False
2094	False	False	False	True	False	False	False	False	False	False	True	False
2095	False	False	False	True	True	True	True	True	True	True	True	True

2096 rows × 12 columns

In [7]:

news.groupby(['author']).count()

Out[7]:

	published	title	text	language	site_url	main_img_url	type	label	title_without_stopwords	text_without_stopwords	hasImage
author
# 1 NWO Hatr	3	3	3	3	3	3	3	3	3	3	3
-NO AUTHOR-	58	58	24	58	58	58	58	58	58	22	58
4 Goals For The Neomasculinity Movement During Trump’s First Term	1	1	1	1	1	1	1	1	1	1	1
?????? ???? ???? ?????????	1	1	1	1	1	1	1	1	1	1	1
A. Griffee	2	2	2	2	2	2	2	2	2	2	2
...	...	...	...	...	...	...	...	...	...	...	...
watchmannonthewall	1	1	1	1	1	1	1	1	1	1	1
willz	1	1	1	1	1	1	1	1	1	1	1
wmw_admin	1	1	1	1	1	1	1	1	1	1	1
wtromp@operamail.com (WT)	2	2	2	2	2	2	2	2	2	2	2
xuenchen	3	3	3	3	3	3	3	3	3	3	3

491 rows × 11 columns

In [8]:

news.groupby(['type']).count()

Out[8]:

	author	published	title	text	language	site_url	main_img_url	label	title_without_stopwords	text_without_stopwords	hasImage
type
bias	436	436	436	393	436	436	436	436	436	389	436
bs	601	601	601	599	601	601	601	601	600	599	601
conspiracy	430	430	430	430	430	430	430	430	430	430	430
fake	15	15	15	15	15	15	15	15	15	15	15
hate	244	244	244	244	244	244	244	244	244	244	244
junksci	102	102	102	102	102	102	102	102	102	102	102
satire	146	146	146	146	146	146	146	146	146	146	146
state	121	121	121	121	121	121	121	121	121	121	121

In [9]:

news.groupby(['language']).count()

Out[9]:

	author	published	title	text	site_url	main_img_url	type	label	title_without_stopwords	text_without_stopwords	hasImage
language
english	2017	2017	2017	1972	2017	2017	2017	2017	2016	1968	2017
french	2	2	2	2	2	2	2	2	2	2	2
german	72	72	72	72	72	72	72	72	72	72	72
ignore	3	3	3	3	3	3	3	3	3	3	3
spanish	1	1	1	1	1	1	1	1	1	1	1

In [ ]:

#https://scikit-learn.org/stable/modules/feature_extraction.html