Print Page - Cleaning up duplicate language strings

Public area => The Pub => Bug reports => Topic started by: Nao on June 11th, 2012, 06:06 PM

Title: Cleaning up duplicate language strings
Post by: Nao on June 11th, 2012, 06:06 PM

So, I made a quick and dirty script to find duplicate keys in the language folder...
There are PLENTY, actually.
And some of them do look like they should be removed.

I'm just going to ask: are there any strings below which you think are safe to remove, and if yes, from which files...?

/Admin.english.php
/Login.english.php

(
[registration_agreement] => Registration Agreement
)

/Admin.english.php
/ManageMembers.english.php

(
[not_activated] => Not activated
[membergroups_add_group] => Add group
)

/Admin.english.php
/ManagePaid.english.php

(
[paid_subs_view] => View Subscriptions
)

/Admin.english.php
/Profile.english.php

(
[display_name] => Display name
[ip_address] => IP address
[date_format] => (YYYY-MM-DD)
[age] => User age
)

/Admin.english.php
/index.english.php

(
[search_for] => Search for
[spiders] => Spiders
)

/Errors.english.php
/Login.english.php

(
[no_password] => Password field empty
)

/Errors.english.php
/Profile.english.php

(
[incorrect_answer] => Sorry, but you did not answer your question correctly. Please click back to try again, or click back twice to use the default method of obtaining your password.
)

/Install.english.php
/index.english.php

(
[lang_rtl] =>
[error_invalid_characters_username] => Invalid character used in Username.
)

/Login.english.php
/Admin.english.php

(
[registration_agreement] => Registration Agreement
)

/Login.english.php
/Errors.english.php

(
[no_password] => You didn't enter your password.
)

/ManageMembers.english.php
/Admin.english.php

(
[membergroups_add_group] => Add group
[not_activated] => Not activated
)

/ManagePaid.english.php
/Admin.english.php

(
[paid_subs_view] => View Subscriptions
)

/ManagePaid.english.php
/Profile.english.php

(
[subscriptions] => Subscriptions
)

/ManagePaid.english.php
/index.english.php

(
[yes] => Yes
[no] => No
)

/ManageSettings.english.php
/PersonalMessage.english.php

(
[pm_read] => Groups allowed to read their messages
)

/ManageSettings.english.php
/Profile.english.php

(
[signature_minposts] => Minimum number of posts for signatures to be visible
)

/Media.english.php
/Post.english.php

(
=> Media
)

/Packages.english.php
/index.english.php

(
[remove] => Remove
)

/PersonalMessage.english.php
/ManageSettings.english.php

(
[pm_read] => Read
)

/PersonalMessage.english.php
/Profile.english.php

(
[pm_display_mode_all] => All at once
[pm_display_mode_one] => One at a time
[pm_display_mode_linked] => As a conversation
[showDrafts] => Show Drafts
[showDrafts_desc] => This section shows you all the draft messages you have saved, or were saved on your behalf.
[show_drafts_none] => You have no draft messages saved at this time.
[edit_draft] => Edit draft
[draftAutoPurge] => (
[1] => Drafts are stored on the server for up to a day, and if not posted or modified in that time, they will be removed.
[n] => Drafts are stored on the server for up to %s days, and if not posted or modified in that time, they will be removed.
)

[remove_all_drafts] => Remove all drafts
[remove_all_drafts_confirm] => Are you sure you want to remove all of your draft messages? (This operation is not reversible.)
)

/Post.english.php
/Media.english.php

(
=> Upload a media file
)

/Post.english.php
/index.english.php

(
[message_icon] => Message icon
[regards_team] => Regards,
The {forum_name} Team.
[change_color] => Change Color
[notify_replies] => Notify me of replies.
[awaiting_approval] => Awaiting approval
)

/Profile.english.php
/Admin.english.php

(
[age] => Age
[date_format] => The format here will be used to show dates throughout this forum.
[display_name] => Display name
[ip_address] => IP address
)

/Profile.english.php
/Errors.english.php

(
[incorrect_answer] => Sorry, but you did not specify a valid combination of Secret Question and Answer in your profile. Please click on the back button, and use the default method of obtaining your password.
)

/Profile.english.php
/ManagePaid.english.php

(
[subscriptions] => Paid Subscriptions
)

/Profile.english.php
/ManageSettings.english.php

(
[signature_minposts] => (
[1] => Your signature will not be shown to other members until you have made at least 1 post.
[n] => Your signature will not be shown to other members until you have made at least %s posts.
)

)

/Profile.english.php
/PersonalMessage.english.php

(
[showDrafts] => Show Drafts
[showDrafts_desc] => This section shows you all the draft posts you have saved, or were saved on your behalf.
[show_drafts_none] => You have no draft messages saved at this time.
[edit_draft] => Edit draft
[draftAutoPurge] => (
[1] => Drafts are stored on the server for up to a day, and if not posted or modified in that time, they will be removed.
[n] => Drafts are stored on the server for up to %s days, and if not posted or modified in that time, they will be removed.
)

[remove_all_drafts] => Remove all drafts
[remove_all_drafts_confirm] => Are you sure you want to remove all of your draft messages? (This operation is not reversible.)
[pm_display_mode_all] => All at once
[pm_display_mode_one] => One at a time
[pm_display_mode_linked] => As a conversation
)

/Profile.english.php
/Reports.english.php

(
[none] => None
)

/Profile.english.php
/index.english.php

(
[hidden] => hidden
[notification] => Notifications
)

/Reports.english.php
/Profile.english.php

(
[none] => None
)

/Search.english.php
/index.english.php

(
[search_date_posted] => Date Posted
[spider] => Spider
)

/index.english.php
/Admin.english.php

(
[search_for] => Search for
[spiders] => Spiders
)

/index.english.php
/Install.english.php

(
[lang_rtl] =>
[error_invalid_characters_username] => Invalid character used in Username.
)

/index.english.php
/ManagePaid.english.php

(
[yes] => Yes
[no] => No
)

/index.english.php
/Packages.english.php

(
[remove] => Remove
)

/index.english.php
/Post.english.php

(
[regards_team] => Regards,
The {forum_name} Team.
[notify_replies] => Notify of replies
[awaiting_approval] => Awaiting Approval
[message_icon] => Message Icon
[change_color] => Change Color
)

/index.english.php
/Profile.english.php

(
[notification] => Notification
[hidden] => Hidden
)

/index.english.php
/Search.english.php

(
[spider] => Spider
[search_date_posted] => Posted:
)

Title: Re: Cleaning up duplicate language strings
Post by: Nao on June 11th, 2012, 06:11 PM

Obviously, some of these strings have no reason to stay (yes/no for ManagePaid...), others have no reason to go (lang_rtl?), others are buggy above (the media tag, uh... if you see an empty key, it's because it's a media tag or something else.)

For the record, here are the EXACT matches between various files (key + content). I think they have the upper priority when it comes to determining what should go:

/Admin.english.php
/Login.english.php

(
[registration_agreement] => Registration Agreement
)

/Admin.english.php
/ManageMembers.english.php

(
[not_activated] => Not activated
[membergroups_add_group] => Add group
)

/Admin.english.php
/ManagePaid.english.php

(
[paid_subs_view] => View Subscriptions
)

/Admin.english.php
/Profile.english.php

(
[display_name] => Display name
[ip_address] => IP address
)

/Admin.english.php
/index.english.php

(
[search_for] => Search for
[spiders] => Spiders
)

/Install.english.php
/index.english.php

(
[lang_rtl] =>
[error_invalid_characters_username] => Invalid character used in Username.
)

/Login.english.php
/Admin.english.php

(
[registration_agreement] => Registration Agreement
)

/ManageMembers.english.php
/Admin.english.php

(
[membergroups_add_group] => Add group
[not_activated] => Not activated
)

/ManagePaid.english.php
/Admin.english.php

(
[paid_subs_view] => View Subscriptions
)

/ManagePaid.english.php
/index.english.php

(
[yes] => Yes
[no] => No
)

/Packages.english.php
/index.english.php

(
[remove] => Remove
)

/PersonalMessage.english.php
/Profile.english.php

(
[pm_display_mode_all] => All at once
[pm_display_mode_one] => One at a time
[pm_display_mode_linked] => As a conversation
[showDrafts] => Show Drafts
[show_drafts_none] => You have no draft messages saved at this time.
[edit_draft] => Edit draft
[draftAutoPurge] => (
[1] => Drafts are stored on the server for up to a day, and if not posted or modified in that time, they will be removed.
[n] => Drafts are stored on the server for up to %s days, and if not posted or modified in that time, they will be removed.
)

[remove_all_drafts] => Remove all drafts
[remove_all_drafts_confirm] => Are you sure you want to remove all of your draft messages? (This operation is not reversible.)
)

/Post.english.php
/index.english.php

(
[regards_team] => Regards,
The {forum_name} Team.
[change_color] => Change Color
)

/Profile.english.php
/Admin.english.php

(
[display_name] => Display name
[ip_address] => IP address
)

/Profile.english.php
/PersonalMessage.english.php

(
[showDrafts] => Show Drafts
[show_drafts_none] => You have no draft messages saved at this time.
[edit_draft] => Edit draft
[draftAutoPurge] => (
[1] => Drafts are stored on the server for up to a day, and if not posted or modified in that time, they will be removed.
[n] => Drafts are stored on the server for up to %s days, and if not posted or modified in that time, they will be removed.
)

[remove_all_drafts] => Remove all drafts
[remove_all_drafts_confirm] => Are you sure you want to remove all of your draft messages? (This operation is not reversible.)
[pm_display_mode_all] => All at once
[pm_display_mode_one] => One at a time
[pm_display_mode_linked] => As a conversation
)

/Profile.english.php
/Reports.english.php

(
[none] => None
)

/Reports.english.php
/Profile.english.php

(
[none] => None
)

/Search.english.php
/index.english.php

(
[spider] => Spider
)

/index.english.php
/Admin.english.php

(
[search_for] => Search for
[spiders] => Spiders
)

/index.english.php
/Install.english.php

(
[lang_rtl] =>
[error_invalid_characters_username] => Invalid character used in Username.
)

/index.english.php
/ManagePaid.english.php

(
[yes] => Yes
[no] => No
)

/index.english.php
/Packages.english.php

(
[remove] => Remove
)

/index.english.php
/Post.english.php

(
[regards_team] => Regards,
The {forum_name} Team.
[change_color] => Change Color
)

/index.english.php
/Search.english.php

(
[spider] => Spider
)

Title: Re: Cleaning up duplicate language strings
Post by: live627 on August 26th, 2013, 02:12 AM

Title: Re: Where should I put up the source code?
Post by: Pandos on October 24th, 2013, 08:25 PM

John mentioned somewhere the elimination of duplicated language strings.
Is there any progress on this? Found a lot in e.g. login.english.php....

Title: Re : Re: Where should I put up the source code?
Post by: live627 on October 25th, 2013, 01:51 AM

I think it was started on, but was never finished.

Title: Re : Re: Where should I put up the source code?
Post by: Nao on October 25th, 2013, 10:27 AM

Yeah, I think I removed a lot, but don't have a tool to find them automatically, so at one point I just went to something else.

TE, are there any tools for that in NetBeans...? :P

PS: are we talking $txt indices, or text content..? Because similar text content, while sometimes it could be simplified into a single string, might be of use as separarate strings, especially for very, very generic words.

Title: Re : Re: Where should I put up the source code?
Post by: Pandos on October 25th, 2013, 01:47 PM

Yes, we're talking about $txt.
Thought there's a solution with awk, but it does'nt work.

So the only option is to delete all text content from files and then search for duplicate entries. Postponed it to later... :)

Title: Re: Cleaning up duplicate language strings
Post by: Nao on October 25th, 2013, 04:23 PM

Moved these posts to the correct topic...

awk is too esoteric for me. ;)
I'm already in enough of a mess, with having to learn the git intricacies over and over again (because I keep forgetting them), so... Let's not push it too far, eh eh... ;)

Still, I quickly cooked up a small script, because I was stupid enough not to post the script I mentioned in the first post, ahum...

Basically, I copied the languages folder to a new place (languages2), and ran this search & replace regex on it:

\$txt\['([^']+)'] = '(.*?)'; to: $txt['$1'] = '';

And then, ran this PHP on that folder:

Code: [Select]

	$arr = array();
	foreach (scandir('./languages2/') as $fil)
	{
		if ($fil == '.' || $fil == '..' || substr($fil, -4) != '.php' || strpos($fil, '.english.') === false)
			continue;
		$fichier = file('./languages2/' . $fil, FILE_SKIP_EMPTY_LINES | FILE_IGNORE_NEW_LINES);
		$arr = array_merge($arr, $fichier);
		echo $fil." (" .count($arr).")\n<br>";
	}
	$b = array_count_values($arr);
	foreach ($b as $key => $val)
		if (strpos($key, '$txt') !== 0 || $val === 1)
			unset($b[$key]);
	arsort($b, SORT_NUMERIC);
	print_r($b);

Remaining duplicate strings, by number of occurrences:

Code: [Select]

    [$txt['plugins_repo_auth'] = '';] => 2
    [$txt['notification'] = '';] => 2
    [$txt['media_admin_bans_mems_empty'] = '';] => 2
    [$txt['lang_rtl'] = false;] => 2
    [$txt['date_format'] = '';] => 2
    [$txt['not_activated'] = '';] => 2
    [$txt['age'] = '';] => 2
    [$txt['no_password'] = '';] => 2
    [$txt['media_admin_modlog_desc'] = '';] => 2
    [$txt['spider'] = '';] => 2
    [$txt['registration_agreement'] = '';] => 2
    [$txt['pm_read'] = '';] => 2
    [$txt['view_all_members'] = '';] => 2
    [$txt['ip_address'] = '';] => 2
    [$txt['display_name'] = '';] => 2
    [$txt['none'] = '';] => 2
    [$txt['membergroups_add_group'] = '';] => 2
    [$txt['showDrafts_desc'] = '';] => 2
    [$txt['show_drafts_none'] = '';] => 2
    [$txt['showDrafts'] = '';] => 2
    [$txt['spiders'] = '';] => 2
    [$txt['error_invalid_characters_username'] = '';] => 2
    [$txt['preset_infractions'] = '';] => 2
    [$txt['edit_draft'] = '';] => 2
    [$txt['draftAutoPurge'] = array(] => 2
    [$txt['ban_type_hostname'] = '';] => 2
    [$txt['ban_invalid_member'] = '';] => 2
    [$txt['ban_type_member_name'] = '';] => 2
    [$txt['subscriptions'] = '';] => 2
    [$txt['remove_all_drafts'] = '';] => 2
    [$txt['remove_all_drafts_confirm'] = '';] => 2
    [$txt['hidden'] = '';] => 2
    [$txt['media'] = '';] => 2
    [$txt['infraction_points'] = '';] => 2

Posted: October 25th, 2013, 04:21 PM

So, basically, that's not a LOT of strings... And some of them really need to be duplicated, such as showDrafts.

Wedge

Public area => The Pub => Bug reports => Topic started by: Nao on June 11th, 2012, 06:06 PM